[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 07:25:10 dist-test-slave-1drq krb5kdc[5760](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 07:25:10 dist-test-slave-1drq krb5kdc[5760](info): set up 2 sockets
Sep 25 07:25:10 dist-test-slave-1drq krb5kdc[5760](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260925 07:25:11.308468 5753 mini_kdc.cc:121] Time spent starting KDC: real 0.968s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Sep 25 07:25:11 dist-test-slave-1drq krb5kdc[5760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321111, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T07:25:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T07:25:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
I20260925 07:25:11.415499 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:39529
--webserver_interface=127.5.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:33137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.158.126:39529
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:25:11.490805 5773 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:25:11.491010 5773 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:25:11.491043 5773 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:25:11.492100 5773 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:25:11.492162 5773 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:25:11.492185 5773 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:25:11.492205 5773 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:25:11.492234 5773 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:25:11.493635 5773 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:33137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.158.126:39529
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:39529
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5773
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:25:11.493911 5773 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:25:11.494112 5773 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:25:11.499190 5781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:11.499137 5779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:11.499109 5773 server_base.cc:1034] running on GCE node
W20260925 07:25:11.499095 5778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:11.499578 5773 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:25:11.499812 5773 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:25:11.500958 5773 hybrid_clock.cc:648] HybridClock initialized: now 1790321111500939 us; error 34 us; skew 500 ppm
Sep 25 07:25:11 dist-test-slave-1drq krb5kdc[5760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321111, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:25:11.503376 5773 init.cc:377] Logged in from keytab as kudu/127.5.158.126@KRBTEST.COM (short username kudu)
I20260925 07:25:11.503803 5773 webserver.cc:460] Webserver started at http://127.5.158.126:41115/ using document root <none> and password file <none>
I20260925 07:25:11.504005 5773 fs_manager.cc:359] Metadata directory not provided
I20260925 07:25:11.504068 5773 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:25:11.504210 5773 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:25:11.505244 5773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/master-0/data/instance:
uuid: "ea244cf669f641858252eca168a2be8a"
format_stamp: "Formatted at 2026-09-25 07:25:11 on dist-test-slave-1drq"
server_key: "b2e5d4f751eee5a5b52be4260f626b83"
server_key_iv: "1aae9635361e16286c25d9e6464e9211"
server_key_version: "encryptionkey@0"
I20260925 07:25:11.505589 5773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/master-0/wal/instance:
uuid: "ea244cf669f641858252eca168a2be8a"
format_stamp: "Formatted at 2026-09-25 07:25:11 on dist-test-slave-1drq"
server_key: "b2e5d4f751eee5a5b52be4260f626b83"
server_key_iv: "1aae9635361e16286c25d9e6464e9211"
server_key_version: "encryptionkey@0"
I20260925 07:25:11.506866 5773 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 07:25:11.507534 5788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:11.507697 5773 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 07:25:11.507794 5773 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "ea244cf669f641858252eca168a2be8a"
format_stamp: "Formatted at 2026-09-25 07:25:11 on dist-test-slave-1drq"
server_key: "b2e5d4f751eee5a5b52be4260f626b83"
server_key_iv: "1aae9635361e16286c25d9e6464e9211"
server_key_version: "encryptionkey@0"
I20260925 07:25:11.507870 5773 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:25:11.558090 5773 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:25:11.560428 5773 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:25:11.560590 5773 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:25:11.564555 5773 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.126:39529
I20260925 07:25:11.564591 5840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.126:39529 every 8 connection(s)
I20260925 07:25:11.564970 5773 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:25:11.565815 5841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:25:11.567876 5841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea244cf669f641858252eca168a2be8a: Bootstrap starting.
I20260925 07:25:11.568382 5841 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea244cf669f641858252eca168a2be8a: Neither blocks nor log segments found. Creating new log.
I20260925 07:25:11.568600 5841 log.cc:824] T 00000000000000000000000000000000 P ea244cf669f641858252eca168a2be8a: Log is configured to *not* fsync() on all Append() calls
I20260925 07:25:11.569340 5841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea244cf669f641858252eca168a2be8a: No bootstrap required, opened a new log
I20260925 07:25:11.570238 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 5773
I20260925 07:25:11.570340 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/master-0/wal/instance
I20260925 07:25:11.571101 5753 external_mini_cluster.cc:1428] Setting key 98cffedd7bc4cf8f9f01ce0c254841a9
I20260925 07:25:11.571411 5841 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea244cf669f641858252eca168a2be8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea244cf669f641858252eca168a2be8a" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 39529 } }
I20260925 07:25:11.571578 5841 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea244cf669f641858252eca168a2be8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:25:11.571614 5841 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea244cf669f641858252eca168a2be8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea244cf669f641858252eca168a2be8a, State: Initialized, Role: FOLLOWER
I20260925 07:25:11.571741 5841 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea244cf669f641858252eca168a2be8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea244cf669f641858252eca168a2be8a" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 39529 } }
I20260925 07:25:11.571817 5841 raft_consensus.cc:388] T 00000000000000000000000000000000 P ea244cf669f641858252eca168a2be8a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 07:25:11.571841 5841 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea244cf669f641858252eca168a2be8a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 07:25:11.571909 5841 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea244cf669f641858252eca168a2be8a [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:25:11.572942 5841 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea244cf669f641858252eca168a2be8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea244cf669f641858252eca168a2be8a" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 39529 } }
I20260925 07:25:11.573052 5841 leader_election.cc:302] T 00000000000000000000000000000000 P ea244cf669f641858252eca168a2be8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ea244cf669f641858252eca168a2be8a; no voters:
I20260925 07:25:11.573225 5841 leader_election.cc:288] T 00000000000000000000000000000000 P ea244cf669f641858252eca168a2be8a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 07:25:11.573302 5846 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea244cf669f641858252eca168a2be8a [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:25:11.573459 5846 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea244cf669f641858252eca168a2be8a [term 1 LEADER]: Becoming Leader. State: Replica: ea244cf669f641858252eca168a2be8a, State: Running, Role: LEADER
I20260925 07:25:11.573462 5841 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea244cf669f641858252eca168a2be8a [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:25:11.573566 5846 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea244cf669f641858252eca168a2be8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea244cf669f641858252eca168a2be8a" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 39529 } }
I20260925 07:25:11.575279 5847 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea244cf669f641858252eca168a2be8a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ea244cf669f641858252eca168a2be8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea244cf669f641858252eca168a2be8a" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 39529 } } }
I20260925 07:25:11.575382 5847 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea244cf669f641858252eca168a2be8a [sys.catalog]: This master's current role is: LEADER
I20260925 07:25:11.575623 5848 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea244cf669f641858252eca168a2be8a [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea244cf669f641858252eca168a2be8a. Latest consensus state: current_term: 1 leader_uuid: "ea244cf669f641858252eca168a2be8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea244cf669f641858252eca168a2be8a" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 39529 } } }
I20260925 07:25:11.575748 5848 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea244cf669f641858252eca168a2be8a [sys.catalog]: This master's current role is: LEADER
I20260925 07:25:11.575853 5861 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 07:25:11.576351 5861 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 07:25:11.578123 5861 catalog_manager.cc:1347] Generated new cluster ID: 5c1ccf5271744ce282144c800b0a4f2e
I20260925 07:25:11.578169 5861 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 07:25:11 dist-test-slave-1drq krb5kdc[5760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321111, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:25:11.588373 5844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:11.572067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54896 (local address 127.5.158.126:39529)
0925 07:25:11.572257 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:11.572266 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:11.572523 (+ 257us) server_negotiation.cc:407] Connection header received
0925 07:25:11.574680 (+ 2157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:11.574686 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:11.574734 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:11.574873 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:11.577159 (+ 2286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:11.578359 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:11.579778 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:11.580329 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:11.580415 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:11.584536 (+ 4121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:11.584540 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:11.584540 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:11.584550 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:11.586478 (+ 1928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:11.586943 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:11.586946 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:11.586946 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:11.587037 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:11.587327 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:11.587328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:11.587328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:11.587596 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:11.587732 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:11.588038 (+ 306us) server_negotiation.cc:299] Negotiation successful
0925 07:25:11.588165 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":81,"threads_started":1}
I20260925 07:25:11.589943 5861 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 07:25:11.590421 5861 catalog_manager.cc:1504] Loading token signing keys...
I20260925 07:25:11.594575 5861 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea244cf669f641858252eca168a2be8a: Generated new TSK 0
I20260925 07:25:11.594735 5861 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.158.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab.
Entry for principal kudu/127.5.158.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab.
I20260925 07:25:11.620468 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.65:0
--local_ip_for_outbound_sockets=127.5.158.65
--webserver_interface=127.5.158.65
--webserver_port=0
--tserver_master_addrs=127.5.158.126:39529
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab
--principal=kudu/127.5.158.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:33137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:25:11.697664 5867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:25:11.697831 5867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:25:11.697853 5867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:25:11.698942 5867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:25:11.698980 5867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:25:11.699007 5867 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.65
I20260925 07:25:11.700389 5867 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:33137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab
--principal=kudu/127.5.158.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.158.65
--webserver_port=0
--tserver_master_addrs=127.5.158.126:39529
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5867
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.65
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:25:11.700610 5867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:25:11.700836 5867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:25:11.705529 5873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:11.705538 5872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:11.705552 5867 server_base.cc:1034] running on GCE node
W20260925 07:25:11.705562 5875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:11.705943 5867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:25:11.706218 5867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:25:11.707383 5867 hybrid_clock.cc:648] HybridClock initialized: now 1790321111707377 us; error 35 us; skew 500 ppm
Sep 25 07:25:11 dist-test-slave-1drq krb5kdc[5760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321111, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:25:11.709830 5867 init.cc:377] Logged in from keytab as kudu/127.5.158.65@KRBTEST.COM (short username kudu)
I20260925 07:25:11.710170 5867 webserver.cc:460] Webserver started at http://127.5.158.65:33575/ using document root <none> and password file <none>
I20260925 07:25:11.710402 5867 fs_manager.cc:359] Metadata directory not provided
I20260925 07:25:11.710444 5867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:25:11.710526 5867 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:25:11.711486 5867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/ts-0/data/instance:
uuid: "7b6daa87799344368bd2e99a383c059d"
format_stamp: "Formatted at 2026-09-25 07:25:11 on dist-test-slave-1drq"
server_key: "26d0427e5e10e0f35665b6ea94913cb4"
server_key_iv: "50b82e4eca7fbd58e5d793670f218cac"
server_key_version: "encryptionkey@0"
I20260925 07:25:11.711771 5867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/ts-0/wal/instance:
uuid: "7b6daa87799344368bd2e99a383c059d"
format_stamp: "Formatted at 2026-09-25 07:25:11 on dist-test-slave-1drq"
server_key: "26d0427e5e10e0f35665b6ea94913cb4"
server_key_iv: "50b82e4eca7fbd58e5d793670f218cac"
server_key_version: "encryptionkey@0"
I20260925 07:25:11.713003 5867 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 07:25:11.713740 5882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:11.713953 5867 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 07:25:11.714032 5867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/ts-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/ts-0/wal
uuid: "7b6daa87799344368bd2e99a383c059d"
format_stamp: "Formatted at 2026-09-25 07:25:11 on dist-test-slave-1drq"
server_key: "26d0427e5e10e0f35665b6ea94913cb4"
server_key_iv: "50b82e4eca7fbd58e5d793670f218cac"
server_key_version: "encryptionkey@0"
I20260925 07:25:11.714083 5867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:25:11.728165 5867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:25:11.729818 5867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:25:11.729924 5867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:25:11.730083 5867 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:25:11.730348 5867 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:25:11.730379 5867 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:11.730398 5867 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:25:11.730444 5867 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:11.735932 5867 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.65:36411
I20260925 07:25:11.736025 5995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.65:36411 every 8 connection(s)
I20260925 07:25:11.736284 5867 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
Sep 25 07:25:11 dist-test-slave-1drq krb5kdc[5760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321111, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.65@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:25:11.744360 5844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:11.736955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:43451 (local address 127.5.158.126:39529)
0925 07:25:11.736993 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:11.736995 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:11.737218 (+ 223us) server_negotiation.cc:407] Connection header received
0925 07:25:11.737629 (+ 411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:11.737630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:11.737640 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:11.737704 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:11.738502 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:11.738972 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:11.739936 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:11.740129 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:11.740181 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:11.742031 (+ 1850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:11.742033 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:11.742033 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:11.742041 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:11.743183 (+ 1142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:11.743529 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:11.743530 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:11.743530 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:11.743580 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:11.743787 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:11.743788 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:11.743788 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:11.743881 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:11.743932 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:11.744153 (+ 221us) server_negotiation.cc:299] Negotiation successful
0925 07:25:11.744240 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260925 07:25:11.744365 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 5867
I20260925 07:25:11.744398 5998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:11.736914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:39529 (local address 127.5.158.65:43451)
0925 07:25:11.737074 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:11.737131 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:11.737552 (+ 421us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:11.737759 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:11.737761 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:11.738266 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:11.738432 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:11.738436 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:11.739044 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:11.739044 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:11.739863 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:11.739865 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:11.740211 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:11.740212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:11.740310 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:11.740700 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:11.740707 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:11.741926 (+ 1219us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:11.743258 (+ 1332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:11.743263 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:11.743265 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:11.743464 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:11.743639 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:11.743640 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:11.743640 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:11.743733 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:11.743935 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:11.743938 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:11.744049 (+ 111us) client_negotiation.cc:769] Sending connection context
0925 07:25:11.744125 (+ 76us) client_negotiation.cc:240] Negotiation successful
0925 07:25:11.744262 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":92,"threads_started":1}
I20260925 07:25:11.744449 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/ts-0/wal/instance
I20260925 07:25:11.744582 5753 external_mini_cluster.cc:1428] Setting key 0cfa6854743acad97c4f9cc0bebb169e
I20260925 07:25:11.744894 5996 heartbeater.cc:347] Connected to a master server at 127.5.158.126:39529
I20260925 07:25:11.744979 5996 heartbeater.cc:464] Registering TS with master...
I20260925 07:25:11.745178 5996 heartbeater.cc:511] Master 127.5.158.126:39529 requested a full tablet report, sending...
I20260925 07:25:11.745736 5805 ts_manager.cc:194] Registered new tserver with Master: 7b6daa87799344368bd2e99a383c059d (127.5.158.65:36411)
I20260925 07:25:11.746594 5805 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.65@KRBTEST.COM'} at 127.5.158.65:43451
WARNING: no policy specified for kudu/127.5.158.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab.
Entry for principal kudu/127.5.158.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab.
I20260925 07:25:11.768389 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.66:0
--local_ip_for_outbound_sockets=127.5.158.66
--webserver_interface=127.5.158.66
--webserver_port=0
--tserver_master_addrs=127.5.158.126:39529
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab
--principal=kudu/127.5.158.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:33137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:25:11.848083 6001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:25:11.848289 6001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:25:11.848321 6001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:25:11.849429 6001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:25:11.849493 6001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:25:11.849536 6001 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.66
I20260925 07:25:11.850839 6001 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:33137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab
--principal=kudu/127.5.158.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.158.66
--webserver_port=0
--tserver_master_addrs=127.5.158.126:39529
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.66
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:25:11.851073 6001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:25:11.851263 6001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:25:11.855584 6009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:11.855563 6007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:11.855669 6001 server_base.cc:1034] running on GCE node
W20260925 07:25:11.855575 6006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:11.855962 6001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:25:11.856179 6001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:25:11.857302 6001 hybrid_clock.cc:648] HybridClock initialized: now 1790321111857285 us; error 43 us; skew 500 ppm
Sep 25 07:25:11 dist-test-slave-1drq krb5kdc[5760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321111, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:25:11.859349 6001 init.cc:377] Logged in from keytab as kudu/127.5.158.66@KRBTEST.COM (short username kudu)
I20260925 07:25:11.859750 6001 webserver.cc:460] Webserver started at http://127.5.158.66:33755/ using document root <none> and password file <none>
I20260925 07:25:11.859944 6001 fs_manager.cc:359] Metadata directory not provided
I20260925 07:25:11.860000 6001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:25:11.860137 6001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:25:11.860942 6001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/ts-1/data/instance:
uuid: "67e7e48c3bc84127a5717ab2700adea0"
format_stamp: "Formatted at 2026-09-25 07:25:11 on dist-test-slave-1drq"
server_key: "0cd7105d477989b1970410ed907fd998"
server_key_iv: "3692f50edfc2747a6c7bf8a84bfa10ec"
server_key_version: "encryptionkey@0"
I20260925 07:25:11.861292 6001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/ts-1/wal/instance:
uuid: "67e7e48c3bc84127a5717ab2700adea0"
format_stamp: "Formatted at 2026-09-25 07:25:11 on dist-test-slave-1drq"
server_key: "0cd7105d477989b1970410ed907fd998"
server_key_iv: "3692f50edfc2747a6c7bf8a84bfa10ec"
server_key_version: "encryptionkey@0"
I20260925 07:25:11.862519 6001 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 07:25:11.863201 6016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:11.863353 6001 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:25:11.863446 6001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/ts-1/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/ts-1/wal
uuid: "67e7e48c3bc84127a5717ab2700adea0"
format_stamp: "Formatted at 2026-09-25 07:25:11 on dist-test-slave-1drq"
server_key: "0cd7105d477989b1970410ed907fd998"
server_key_iv: "3692f50edfc2747a6c7bf8a84bfa10ec"
server_key_version: "encryptionkey@0"
I20260925 07:25:11.863517 6001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:25:11.879289 6001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:25:11.881080 6001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:25:11.881240 6001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:25:11.881424 6001 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:25:11.881673 6001 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:25:11.881722 6001 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:11.881752 6001 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:25:11.881775 6001 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:11.887305 6001 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.66:42613
I20260925 07:25:11.887343 6129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.66:42613 every 8 connection(s)
I20260925 07:25:11.887635 6001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
Sep 25 07:25:11 dist-test-slave-1drq krb5kdc[5760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321111, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.66@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:25:11.895562 5844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:11.888394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:54505 (local address 127.5.158.126:39529)
0925 07:25:11.888461 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:11.888462 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:11.888627 (+ 165us) server_negotiation.cc:407] Connection header received
0925 07:25:11.889035 (+ 408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:11.889036 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:11.889044 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:11.889163 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:11.889943 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:11.890462 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:11.891318 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:11.891493 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:11.891542 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:11.893303 (+ 1761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:11.893305 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:11.893305 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:11.893312 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:11.894414 (+ 1102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:11.894736 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:11.894736 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:11.894737 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:11.894783 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:11.895021 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:11.895022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:11.895022 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:11.895113 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:11.895185 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:11.895402 (+ 217us) server_negotiation.cc:299] Negotiation successful
0925 07:25:11.895484 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260925 07:25:11.895591 6132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:11.888372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:39529 (local address 127.5.158.66:54505)
0925 07:25:11.888506 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:11.888542 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:11.888959 (+ 417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:11.889191 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:11.889194 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:11.889695 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:11.889869 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:11.889873 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:11.890535 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:11.890535 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:11.891247 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:11.891248 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:11.891559 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:11.891559 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:11.891646 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:11.892017 (+ 371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:11.892024 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:11.893230 (+ 1206us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:11.894480 (+ 1250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:11.894485 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:11.894486 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:11.894675 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:11.894866 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:11.894867 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:11.894867 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:11.894969 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:11.895179 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:11.895182 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:11.895277 (+ 95us) client_negotiation.cc:769] Sending connection context
0925 07:25:11.895354 (+ 77us) client_negotiation.cc:240] Negotiation successful
0925 07:25:11.895455 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":72,"threads_started":1}
I20260925 07:25:11.895802 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 6001
I20260925 07:25:11.895900 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/ts-1/wal/instance
I20260925 07:25:11.896118 5753 external_mini_cluster.cc:1428] Setting key 26fd3a776d53a39bbd2e3ac7ba55f3b2
I20260925 07:25:11.896270 6130 heartbeater.cc:347] Connected to a master server at 127.5.158.126:39529
I20260925 07:25:11.896356 6130 heartbeater.cc:464] Registering TS with master...
I20260925 07:25:11.896521 6130 heartbeater.cc:511] Master 127.5.158.126:39529 requested a full tablet report, sending...
I20260925 07:25:11.896911 5805 ts_manager.cc:194] Registered new tserver with Master: 67e7e48c3bc84127a5717ab2700adea0 (127.5.158.66:42613)
I20260925 07:25:11.897406 5805 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.66@KRBTEST.COM'} at 127.5.158.66:54505
WARNING: no policy specified for kudu/127.5.158.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab.
Entry for principal kudu/127.5.158.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab.
I20260925 07:25:11.919474 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.67:0
--local_ip_for_outbound_sockets=127.5.158.67
--webserver_interface=127.5.158.67
--webserver_port=0
--tserver_master_addrs=127.5.158.126:39529
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab
--principal=kudu/127.5.158.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:33137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:25:11.991056 6135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:25:11.991202 6135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:25:11.991222 6135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:25:11.992167 6135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:25:11.992197 6135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:25:11.992219 6135 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.67
I20260925 07:25:11.993553 6135 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:33137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab
--principal=kudu/127.5.158.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.158.67
--webserver_port=0
--tserver_master_addrs=127.5.158.126:39529
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6135
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.67
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:25:11.993706 6135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:25:11.993840 6135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:25:11.998275 6141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:11.998284 6143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:11.998462 6135 server_base.cc:1034] running on GCE node
W20260925 07:25:11.998490 6140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:11.998688 6135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:25:11.998898 6135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:25:12.000061 6135 hybrid_clock.cc:648] HybridClock initialized: now 1790321112000053 us; error 39 us; skew 500 ppm
Sep 25 07:25:11 dist-test-slave-1drq krb5kdc[5760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321111, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:25:12.002480 6135 init.cc:377] Logged in from keytab as kudu/127.5.158.67@KRBTEST.COM (short username kudu)
I20260925 07:25:12.002862 6135 webserver.cc:460] Webserver started at http://127.5.158.67:45985/ using document root <none> and password file <none>
I20260925 07:25:12.003074 6135 fs_manager.cc:359] Metadata directory not provided
I20260925 07:25:12.003137 6135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:25:12.003275 6135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:25:12.004204 6135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/ts-2/data/instance:
uuid: "4ce9b9f5dc1942ceb90b30c88d5d7ae4"
format_stamp: "Formatted at 2026-09-25 07:25:11 on dist-test-slave-1drq"
server_key: "5d1488848de23a16a35f572cd89810dd"
server_key_iv: "b2e8a2801a0e5eb2203361bdc6175325"
server_key_version: "encryptionkey@0"
I20260925 07:25:12.004520 6135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/ts-2/wal/instance:
uuid: "4ce9b9f5dc1942ceb90b30c88d5d7ae4"
format_stamp: "Formatted at 2026-09-25 07:25:11 on dist-test-slave-1drq"
server_key: "5d1488848de23a16a35f572cd89810dd"
server_key_iv: "b2e8a2801a0e5eb2203361bdc6175325"
server_key_version: "encryptionkey@0"
I20260925 07:25:12.005815 6135 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 07:25:12.006481 6150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:12.006646 6135 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 07:25:12.006736 6135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/ts-2/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/ts-2/wal
uuid: "4ce9b9f5dc1942ceb90b30c88d5d7ae4"
format_stamp: "Formatted at 2026-09-25 07:25:11 on dist-test-slave-1drq"
server_key: "5d1488848de23a16a35f572cd89810dd"
server_key_iv: "b2e8a2801a0e5eb2203361bdc6175325"
server_key_version: "encryptionkey@0"
I20260925 07:25:12.006808 6135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:25:12.029459 6135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:25:12.031452 6135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:25:12.031584 6135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:25:12.031764 6135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:25:12.032033 6135 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:25:12.032080 6135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:12.032115 6135 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:25:12.032140 6135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:12.037539 6135 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.67:33379
I20260925 07:25:12.037603 6263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.67:33379 every 8 connection(s)
I20260925 07:25:12.037875 6135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
Sep 25 07:25:12 dist-test-slave-1drq krb5kdc[5760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321111, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.67@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:25:12.043232 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 6135
I20260925 07:25:12.043322 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790321110323796-5753-0/minicluster-data/ts-2/wal/instance
I20260925 07:25:12.043480 5753 external_mini_cluster.cc:1428] Setting key 773ea2aea7c8103c89757d06f2b23af7
I20260925 07:25:12.045555 5844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:12.038479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:49065 (local address 127.5.158.126:39529)
0925 07:25:12.038540 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:12.038542 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:12.038682 (+ 140us) server_negotiation.cc:407] Connection header received
0925 07:25:12.039099 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:12.039100 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:12.039108 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:12.039170 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:12.039949 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:12.040366 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:12.041206 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:12.041372 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:12.041414 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:12.043130 (+ 1716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:12.043132 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:12.043132 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:12.043140 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:12.044471 (+ 1331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:12.044805 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:12.044806 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:12.044806 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:12.044851 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:12.045053 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:12.045053 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:12.045053 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:12.045199 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:12.045250 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:12.045439 (+ 189us) server_negotiation.cc:299] Negotiation successful
0925 07:25:12.045496 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260925 07:25:12.045768 6266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:12.038447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:39529 (local address 127.5.158.67:49065)
0925 07:25:12.038575 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:12.038609 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:12.039026 (+ 417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:12.039249 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:12.039252 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:12.039723 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:12.039876 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:12.039879 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:12.040430 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:12.040430 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:12.041139 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:12.041140 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:12.041432 (+ 292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:12.041433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:12.041529 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:12.041922 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:12.041928 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:12.043049 (+ 1121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:12.044535 (+ 1486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:12.044540 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:12.044542 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:12.044745 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:12.044926 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:12.044926 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:12.044926 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:12.045005 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:12.045242 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:12.045245 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:12.045341 (+ 96us) client_negotiation.cc:769] Sending connection context
0925 07:25:12.045423 (+ 82us) client_negotiation.cc:240] Negotiation successful
0925 07:25:12.045529 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":71,"threads_started":1}
I20260925 07:25:12.046249 6264 heartbeater.cc:347] Connected to a master server at 127.5.158.126:39529
I20260925 07:25:12.046324 6264 heartbeater.cc:464] Registering TS with master...
I20260925 07:25:12.046489 6264 heartbeater.cc:511] Master 127.5.158.126:39529 requested a full tablet report, sending...
I20260925 07:25:12.046809 5805 ts_manager.cc:194] Registered new tserver with Master: 4ce9b9f5dc1942ceb90b30c88d5d7ae4 (127.5.158.67:33379)
I20260925 07:25:12.047156 5805 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.67@KRBTEST.COM'} at 127.5.158.67:49065
I20260925 07:25:12.055374 5753 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 25 07:25:12 dist-test-slave-1drq krb5kdc[5760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321112, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 07:25:12 dist-test-slave-1drq krb5kdc[5760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321112, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.158.65@KRBTEST.COM
I20260925 07:25:12.076778 6271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:12.069493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60030 (local address 127.5.158.65:36411)
0925 07:25:12.069650 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:12.069654 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:12.069664 (+ 10us) server_negotiation.cc:407] Connection header received
0925 07:25:12.069686 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:12.069688 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:12.069700 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:12.069772 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:12.070302 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:12.071021 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:12.071678 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:12.071976 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:12.072028 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:12.074180 (+ 2152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:12.074186 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:12.074187 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:12.074194 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:12.075757 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:12.076073 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:12.076075 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:12.076075 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:12.076128 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:12.076275 (+ 147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:12.076276 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:12.076276 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:12.076388 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:12.076477 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:12.076540 (+ 63us) server_negotiation.cc:299] Negotiation successful
0925 07:25:12.076601 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":71,"threads_started":1}
W20260925 07:25:12.077143 5910 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60030
Sep 25 07:25:12 dist-test-slave-1drq krb5kdc[5760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321112, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 07:25:12 dist-test-slave-1drq krb5kdc[5760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321112, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.158.65@KRBTEST.COM
I20260925 07:25:12.096534 6271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:12.090399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60044 (local address 127.5.158.65:36411)
0925 07:25:12.090456 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:12.090458 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:12.090535 (+ 77us) server_negotiation.cc:407] Connection header received
0925 07:25:12.090600 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:12.090601 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:12.090612 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:12.090672 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:12.091104 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:12.091502 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:12.092131 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:12.092305 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:12.092379 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:12.094201 (+ 1822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:12.094202 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:12.094202 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:12.094210 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:12.095452 (+ 1242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:12.095822 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:12.095823 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:12.095823 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:12.095882 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:12.096096 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:12.096097 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:12.096097 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:12.096180 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:12.096287 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:12.096371 (+ 84us) server_negotiation.cc:299] Negotiation successful
0925 07:25:12.096440 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260925 07:25:12.096985 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 5867
I20260925 07:25:12.101032 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 6001
I20260925 07:25:12.104933 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 6135
I20260925 07:25:12.108783 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 5773
2026-09-25T07:25:12Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1792 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 07:25:12 dist-test-slave-1drq krb5kdc[6278](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 07:25:12 dist-test-slave-1drq krb5kdc[6278](info): set up 2 sockets
Sep 25 07:25:12 dist-test-slave-1drq krb5kdc[6278](info): commencing operation
krb5kdc: starting...
W20260925 07:25:13.040418 5753 mini_kdc.cc:121] Time spent starting KDC: real 0.913s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Sep 25 07:25:13 dist-test-slave-1drq krb5kdc[6278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321113, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T07:25:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T07:25:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
I20260925 07:25:13.139684 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:41335
--webserver_interface=127.5.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:42113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.158.126:41335
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:25:13.209118 6291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:25:13.209273 6291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:25:13.209306 6291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:25:13.210281 6291 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:25:13.210318 6291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:25:13.210331 6291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:25:13.210376 6291 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:25:13.210397 6291 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:25:13.211665 6291 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:42113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.158.126:41335
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:41335
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6291
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:25:13.211807 6291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:25:13.212013 6291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:25:13.216049 6299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:13.216089 6291 server_base.cc:1034] running on GCE node
W20260925 07:25:13.216060 6296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:13.216112 6297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:13.216486 6291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:25:13.216737 6291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:25:13.217908 6291 hybrid_clock.cc:648] HybridClock initialized: now 1790321113217906 us; error 31 us; skew 500 ppm
Sep 25 07:25:13 dist-test-slave-1drq krb5kdc[6278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321113, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:25:13.223495 6291 init.cc:377] Logged in from keytab as kudu/127.5.158.126@KRBTEST.COM (short username kudu)
I20260925 07:25:13.223935 6291 webserver.cc:460] Webserver started at http://127.5.158.126:35425/ using document root <none> and password file <none>
I20260925 07:25:13.224156 6291 fs_manager.cc:359] Metadata directory not provided
I20260925 07:25:13.224218 6291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:25:13.224359 6291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:25:13.225351 6291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/master-0/data/instance:
uuid: "80d25ea07fae4ecda76cee3f09628e7a"
format_stamp: "Formatted at 2026-09-25 07:25:13 on dist-test-slave-1drq"
server_key: "f6c4d97cb0255ce70aa81a72a84f7666"
server_key_iv: "8ea0017e251022a14b7186b8a5ac9295"
server_key_version: "encryptionkey@0"
I20260925 07:25:13.225677 6291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/master-0/wal/instance:
uuid: "80d25ea07fae4ecda76cee3f09628e7a"
format_stamp: "Formatted at 2026-09-25 07:25:13 on dist-test-slave-1drq"
server_key: "f6c4d97cb0255ce70aa81a72a84f7666"
server_key_iv: "8ea0017e251022a14b7186b8a5ac9295"
server_key_version: "encryptionkey@0"
I20260925 07:25:13.226845 6291 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 07:25:13.227451 6306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:13.227597 6291 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:25:13.227684 6291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "80d25ea07fae4ecda76cee3f09628e7a"
format_stamp: "Formatted at 2026-09-25 07:25:13 on dist-test-slave-1drq"
server_key: "f6c4d97cb0255ce70aa81a72a84f7666"
server_key_iv: "8ea0017e251022a14b7186b8a5ac9295"
server_key_version: "encryptionkey@0"
I20260925 07:25:13.227748 6291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:25:13.238956 6291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:25:13.240496 6291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:25:13.240583 6291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:25:13.244005 6291 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.126:41335
I20260925 07:25:13.244052 6358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.126:41335 every 8 connection(s)
I20260925 07:25:13.244310 6291 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:25:13.244972 6359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:25:13.247104 6359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80d25ea07fae4ecda76cee3f09628e7a: Bootstrap starting.
I20260925 07:25:13.247736 6359 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 80d25ea07fae4ecda76cee3f09628e7a: Neither blocks nor log segments found. Creating new log.
I20260925 07:25:13.248018 6359 log.cc:824] T 00000000000000000000000000000000 P 80d25ea07fae4ecda76cee3f09628e7a: Log is configured to *not* fsync() on all Append() calls
I20260925 07:25:13.248610 6359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80d25ea07fae4ecda76cee3f09628e7a: No bootstrap required, opened a new log
I20260925 07:25:13.250015 6359 raft_consensus.cc:348] T 00000000000000000000000000000000 P 80d25ea07fae4ecda76cee3f09628e7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80d25ea07fae4ecda76cee3f09628e7a" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 41335 } }
I20260925 07:25:13.250126 6359 raft_consensus.cc:374] T 00000000000000000000000000000000 P 80d25ea07fae4ecda76cee3f09628e7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:25:13.250145 6359 raft_consensus.cc:729] T 00000000000000000000000000000000 P 80d25ea07fae4ecda76cee3f09628e7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80d25ea07fae4ecda76cee3f09628e7a, State: Initialized, Role: FOLLOWER
I20260925 07:25:13.250308 6359 consensus_queue.cc:260] T 00000000000000000000000000000000 P 80d25ea07fae4ecda76cee3f09628e7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80d25ea07fae4ecda76cee3f09628e7a" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 41335 } }
I20260925 07:25:13.250372 6359 raft_consensus.cc:388] T 00000000000000000000000000000000 P 80d25ea07fae4ecda76cee3f09628e7a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 07:25:13.250407 6359 raft_consensus.cc:482] T 00000000000000000000000000000000 P 80d25ea07fae4ecda76cee3f09628e7a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 07:25:13.250449 6359 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 80d25ea07fae4ecda76cee3f09628e7a [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:25:13.250880 6359 raft_consensus.cc:504] T 00000000000000000000000000000000 P 80d25ea07fae4ecda76cee3f09628e7a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80d25ea07fae4ecda76cee3f09628e7a" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 41335 } }
I20260925 07:25:13.250977 6359 leader_election.cc:302] T 00000000000000000000000000000000 P 80d25ea07fae4ecda76cee3f09628e7a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 80d25ea07fae4ecda76cee3f09628e7a; no voters:
I20260925 07:25:13.251130 6359 leader_election.cc:288] T 00000000000000000000000000000000 P 80d25ea07fae4ecda76cee3f09628e7a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 07:25:13.251158 6362 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 80d25ea07fae4ecda76cee3f09628e7a [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:25:13.251305 6362 raft_consensus.cc:686] T 00000000000000000000000000000000 P 80d25ea07fae4ecda76cee3f09628e7a [term 1 LEADER]: Becoming Leader. State: Replica: 80d25ea07fae4ecda76cee3f09628e7a, State: Running, Role: LEADER
I20260925 07:25:13.251338 6359 sys_catalog.cc:564] T 00000000000000000000000000000000 P 80d25ea07fae4ecda76cee3f09628e7a [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:25:13.251395 6362 consensus_queue.cc:237] T 00000000000000000000000000000000 P 80d25ea07fae4ecda76cee3f09628e7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80d25ea07fae4ecda76cee3f09628e7a" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 41335 } }
I20260925 07:25:13.251725 6364 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80d25ea07fae4ecda76cee3f09628e7a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 80d25ea07fae4ecda76cee3f09628e7a. Latest consensus state: current_term: 1 leader_uuid: "80d25ea07fae4ecda76cee3f09628e7a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80d25ea07fae4ecda76cee3f09628e7a" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 41335 } } }
I20260925 07:25:13.251834 6364 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80d25ea07fae4ecda76cee3f09628e7a [sys.catalog]: This master's current role is: LEADER
I20260925 07:25:13.251721 6363 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80d25ea07fae4ecda76cee3f09628e7a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "80d25ea07fae4ecda76cee3f09628e7a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80d25ea07fae4ecda76cee3f09628e7a" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 41335 } } }
I20260925 07:25:13.251947 6363 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80d25ea07fae4ecda76cee3f09628e7a [sys.catalog]: This master's current role is: LEADER
I20260925 07:25:13.252285 6366 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 07:25:13.252724 6366 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 07:25:13.253527 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 6291
I20260925 07:25:13.253603 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/master-0/wal/instance
I20260925 07:25:13.253746 5753 external_mini_cluster.cc:1428] Setting key dceef3569a0f76cd2082305882655c4c
I20260925 07:25:13.254247 6366 catalog_manager.cc:1347] Generated new cluster ID: c587a56bd37b44b9b3ca229d69a6ac17
I20260925 07:25:13.254323 6366 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 07:25:13.260794 6366 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 07:25:13.261457 6366 catalog_manager.cc:1504] Loading token signing keys...
Sep 25 07:25:13 dist-test-slave-1drq krb5kdc[6278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321113, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:25:13.268138 6382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:13.254275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49726 (local address 127.5.158.126:41335)
0925 07:25:13.260678 (+ 6403us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:13.260686 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:13.260715 (+ 29us) server_negotiation.cc:407] Connection header received
0925 07:25:13.260810 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:13.260818 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:13.260888 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:13.261022 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:13.261609 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:13.262347 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:13.263006 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:13.263328 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:13.263426 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:13.265397 (+ 1971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:13.265400 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:13.265401 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:13.265411 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:13.266895 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:13.267212 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:13.267215 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:13.267215 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:13.267290 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:13.267475 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:13.267475 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:13.267476 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:13.267657 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:13.267779 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:13.267878 (+ 99us) server_negotiation.cc:299] Negotiation successful
0925 07:25:13.268000 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6359,"thread_start_us":55,"threads_started":1}
I20260925 07:25:13.269186 6366 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 80d25ea07fae4ecda76cee3f09628e7a: Generated new TSK 0
I20260925 07:25:13.269310 6366 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.158.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab.
Entry for principal kudu/127.5.158.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab.
I20260925 07:25:13.296052 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.65:0
--local_ip_for_outbound_sockets=127.5.158.65
--webserver_interface=127.5.158.65
--webserver_port=0
--tserver_master_addrs=127.5.158.126:41335
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab
--principal=kudu/127.5.158.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:42113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:25:13.395789 6385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:25:13.395977 6385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:25:13.396000 6385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:25:13.397105 6385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:25:13.397164 6385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:25:13.397203 6385 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.65
I20260925 07:25:13.398615 6385 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:42113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab
--principal=kudu/127.5.158.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.158.65
--webserver_port=0
--tserver_master_addrs=127.5.158.126:41335
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6385
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.65
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:25:13.398882 6385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:25:13.399096 6385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:25:13.404564 6390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:13.404577 6393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:13.404577 6391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:13.404722 6385 server_base.cc:1034] running on GCE node
I20260925 07:25:13.405004 6385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:25:13.405308 6385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:25:13.406454 6385 hybrid_clock.cc:648] HybridClock initialized: now 1790321113406444 us; error 28 us; skew 500 ppm
Sep 25 07:25:13 dist-test-slave-1drq krb5kdc[6278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321113, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:25:13.408756 6385 init.cc:377] Logged in from keytab as kudu/127.5.158.65@KRBTEST.COM (short username kudu)
I20260925 07:25:13.409256 6385 webserver.cc:460] Webserver started at http://127.5.158.65:36907/ using document root <none> and password file <none>
I20260925 07:25:13.409467 6385 fs_manager.cc:359] Metadata directory not provided
I20260925 07:25:13.409530 6385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:25:13.409657 6385 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:25:13.410591 6385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/ts-0/data/instance:
uuid: "f162dd4c688b4f8890da9e6f768f6f08"
format_stamp: "Formatted at 2026-09-25 07:25:13 on dist-test-slave-1drq"
server_key: "92c7185f71f82aae4394d563eb9886c3"
server_key_iv: "1f7c2433e5920ad6bd5efdbf11fd3554"
server_key_version: "encryptionkey@0"
I20260925 07:25:13.410972 6385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/ts-0/wal/instance:
uuid: "f162dd4c688b4f8890da9e6f768f6f08"
format_stamp: "Formatted at 2026-09-25 07:25:13 on dist-test-slave-1drq"
server_key: "92c7185f71f82aae4394d563eb9886c3"
server_key_iv: "1f7c2433e5920ad6bd5efdbf11fd3554"
server_key_version: "encryptionkey@0"
I20260925 07:25:13.412204 6385 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 07:25:13.412793 6400 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:13.412971 6385 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 07:25:13.413057 6385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/ts-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/ts-0/wal
uuid: "f162dd4c688b4f8890da9e6f768f6f08"
format_stamp: "Formatted at 2026-09-25 07:25:13 on dist-test-slave-1drq"
server_key: "92c7185f71f82aae4394d563eb9886c3"
server_key_iv: "1f7c2433e5920ad6bd5efdbf11fd3554"
server_key_version: "encryptionkey@0"
I20260925 07:25:13.413159 6385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:25:13.424885 6385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:25:13.426954 6385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:25:13.427089 6385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:25:13.427271 6385 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:25:13.427529 6385 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:25:13.427577 6385 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:13.427613 6385 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:25:13.427639 6385 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:13.433457 6385 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.65:35467
I20260925 07:25:13.433490 6513 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.65:35467 every 8 connection(s)
I20260925 07:25:13.433769 6385 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
Sep 25 07:25:13 dist-test-slave-1drq krb5kdc[6278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321113, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.65@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:25:13.440513 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 6385
I20260925 07:25:13.440613 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/ts-0/wal/instance
I20260925 07:25:13.440781 5753 external_mini_cluster.cc:1428] Setting key b8ed32755bd2008469beff49c1b2ace9
I20260925 07:25:13.442588 6516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:13.434432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:41335 (local address 127.5.158.65:37125)
0925 07:25:13.434581 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:13.434619 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:13.435450 (+ 831us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:13.435639 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:13.435641 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:13.435859 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:13.436021 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:13.436025 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:13.436654 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:13.436655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:13.437498 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:13.437500 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:13.437815 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:13.437816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:13.437908 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:13.438366 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:13.438373 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:13.439582 (+ 1209us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:13.441159 (+ 1577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:13.441166 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:13.441167 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:13.441564 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:13.441799 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:13.441799 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:13.441800 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:13.441902 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:13.442181 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:13.442185 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:13.442279 (+ 94us) client_negotiation.cc:769] Sending connection context
0925 07:25:13.442351 (+ 72us) client_negotiation.cc:240] Negotiation successful
0925 07:25:13.442457 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":85,"threads_started":1}
I20260925 07:25:13.442623 6382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:13.434437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:37125 (local address 127.5.158.126:41335)
0925 07:25:13.434496 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:13.434498 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:13.434697 (+ 199us) server_negotiation.cc:407] Connection header received
0925 07:25:13.435529 (+ 832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:13.435530 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:13.435539 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:13.435592 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:13.436093 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:13.436568 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:13.437566 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:13.437745 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:13.437797 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:13.439662 (+ 1865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:13.439663 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:13.439663 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:13.439669 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:13.441049 (+ 1380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:13.441657 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:13.441658 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:13.441659 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:13.441726 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:13.441968 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:13.441969 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:13.441970 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:13.442117 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:13.442188 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:13.442397 (+ 209us) server_negotiation.cc:299] Negotiation successful
0925 07:25:13.442486 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260925 07:25:13.443411 6514 heartbeater.cc:347] Connected to a master server at 127.5.158.126:41335
I20260925 07:25:13.443495 6514 heartbeater.cc:464] Registering TS with master...
I20260925 07:25:13.443681 6514 heartbeater.cc:511] Master 127.5.158.126:41335 requested a full tablet report, sending...
I20260925 07:25:13.444198 6323 ts_manager.cc:194] Registered new tserver with Master: f162dd4c688b4f8890da9e6f768f6f08 (127.5.158.65:35467)
I20260925 07:25:13.445114 6323 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.65@KRBTEST.COM'} at 127.5.158.65:37125
WARNING: no policy specified for kudu/127.5.158.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab.
Entry for principal kudu/127.5.158.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab.
I20260925 07:25:13.463354 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.66:0
--local_ip_for_outbound_sockets=127.5.158.66
--webserver_interface=127.5.158.66
--webserver_port=0
--tserver_master_addrs=127.5.158.126:41335
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab
--principal=kudu/127.5.158.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:42113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:25:13.538753 6519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:25:13.538930 6519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:25:13.538954 6519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:25:13.539974 6519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:25:13.540025 6519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:25:13.540050 6519 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.66
I20260925 07:25:13.541421 6519 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:42113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab
--principal=kudu/127.5.158.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.158.66
--webserver_port=0
--tserver_master_addrs=127.5.158.126:41335
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6519
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.66
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:25:13.541580 6519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:25:13.541726 6519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:25:13.546325 6524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:13.546308 6527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:13.546308 6525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:13.546510 6519 server_base.cc:1034] running on GCE node
I20260925 07:25:13.546780 6519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:25:13.546998 6519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:25:13.548132 6519 hybrid_clock.cc:648] HybridClock initialized: now 1790321113548119 us; error 29 us; skew 500 ppm
Sep 25 07:25:13 dist-test-slave-1drq krb5kdc[6278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321113, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:25:13.550406 6519 init.cc:377] Logged in from keytab as kudu/127.5.158.66@KRBTEST.COM (short username kudu)
I20260925 07:25:13.550882 6519 webserver.cc:460] Webserver started at http://127.5.158.66:34305/ using document root <none> and password file <none>
I20260925 07:25:13.551091 6519 fs_manager.cc:359] Metadata directory not provided
I20260925 07:25:13.551132 6519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:25:13.551263 6519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:25:13.552248 6519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/ts-1/data/instance:
uuid: "bdf6fab5ba1c4dc08f5a2520ac770951"
format_stamp: "Formatted at 2026-09-25 07:25:13 on dist-test-slave-1drq"
server_key: "3331082b8583eecdf5b4574e684dcda9"
server_key_iv: "9e4f4edf1357c8c94c9de2a1d66e79c9"
server_key_version: "encryptionkey@0"
I20260925 07:25:13.552613 6519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/ts-1/wal/instance:
uuid: "bdf6fab5ba1c4dc08f5a2520ac770951"
format_stamp: "Formatted at 2026-09-25 07:25:13 on dist-test-slave-1drq"
server_key: "3331082b8583eecdf5b4574e684dcda9"
server_key_iv: "9e4f4edf1357c8c94c9de2a1d66e79c9"
server_key_version: "encryptionkey@0"
I20260925 07:25:13.553994 6519 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 07:25:13.554692 6534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:13.554876 6519 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:25:13.554960 6519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/ts-1/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/ts-1/wal
uuid: "bdf6fab5ba1c4dc08f5a2520ac770951"
format_stamp: "Formatted at 2026-09-25 07:25:13 on dist-test-slave-1drq"
server_key: "3331082b8583eecdf5b4574e684dcda9"
server_key_iv: "9e4f4edf1357c8c94c9de2a1d66e79c9"
server_key_version: "encryptionkey@0"
I20260925 07:25:13.555025 6519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:25:13.569275 6519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:25:13.570889 6519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:25:13.571038 6519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:25:13.571249 6519 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:25:13.571540 6519 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:25:13.571592 6519 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:13.571623 6519 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:25:13.571650 6519 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:13.577069 6519 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.66:34615
I20260925 07:25:13.577129 6647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.66:34615 every 8 connection(s)
I20260925 07:25:13.577395 6519 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
Sep 25 07:25:13 dist-test-slave-1drq krb5kdc[6278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321113, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.66@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:25:13.585875 6382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:13.578140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:37279 (local address 127.5.158.126:41335)
0925 07:25:13.578204 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:13.578206 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:13.578430 (+ 224us) server_negotiation.cc:407] Connection header received
0925 07:25:13.579405 (+ 975us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:13.579406 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:13.579419 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:13.579469 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:13.580019 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:13.580499 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:13.581435 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:13.581631 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:13.581710 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:13.583452 (+ 1742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:13.583454 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:13.583454 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:13.583462 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:13.584583 (+ 1121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:13.584947 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:13.584948 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:13.584949 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:13.585018 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:13.585276 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:13.585277 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:13.585278 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:13.585391 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:13.585466 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:13.585712 (+ 246us) server_negotiation.cc:299] Negotiation successful
0925 07:25:13.585785 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 07:25:13.586035 6650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:13.578115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:41335 (local address 127.5.158.66:37279)
0925 07:25:13.578257 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:13.578287 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:13.579288 (+ 1001us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:13.579542 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:13.579545 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:13.579762 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:13.579925 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:13.579928 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:13.580618 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:13.580619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:13.581350 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:13.581352 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:13.581719 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:13.581720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:13.581823 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:13.582256 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:13.582263 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:13.583365 (+ 1102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:13.584665 (+ 1300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:13.584670 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:13.584673 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:13.584874 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:13.585108 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:13.585109 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:13.585109 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:13.585209 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:13.585465 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:13.585471 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:13.585591 (+ 120us) client_negotiation.cc:769] Sending connection context
0925 07:25:13.585691 (+ 100us) client_negotiation.cc:240] Negotiation successful
0925 07:25:13.585831 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":60,"threads_started":1}
I20260925 07:25:13.586503 6648 heartbeater.cc:347] Connected to a master server at 127.5.158.126:41335
I20260925 07:25:13.586570 6648 heartbeater.cc:464] Registering TS with master...
I20260925 07:25:13.586735 6648 heartbeater.cc:511] Master 127.5.158.126:41335 requested a full tablet report, sending...
I20260925 07:25:13.587077 6323 ts_manager.cc:194] Registered new tserver with Master: bdf6fab5ba1c4dc08f5a2520ac770951 (127.5.158.66:34615)
I20260925 07:25:13.587143 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 6519
I20260925 07:25:13.587247 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/ts-1/wal/instance
I20260925 07:25:13.587394 5753 external_mini_cluster.cc:1428] Setting key 191b2201afa9c4e7df9e7d644267e783
I20260925 07:25:13.587545 6323 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.66@KRBTEST.COM'} at 127.5.158.66:37279
WARNING: no policy specified for kudu/127.5.158.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab.
Entry for principal kudu/127.5.158.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab.
I20260925 07:25:13.608958 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.67:0
--local_ip_for_outbound_sockets=127.5.158.67
--webserver_interface=127.5.158.67
--webserver_port=0
--tserver_master_addrs=127.5.158.126:41335
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab
--principal=kudu/127.5.158.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:42113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:25:13.680408 6653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:25:13.680609 6653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:25:13.680644 6653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:25:13.681639 6653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:25:13.681690 6653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:25:13.681730 6653 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.67
I20260925 07:25:13.682940 6653 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:42113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab
--principal=kudu/127.5.158.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.158.67
--webserver_port=0
--tserver_master_addrs=127.5.158.126:41335
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6653
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.67
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:25:13.683145 6653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:25:13.683320 6653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 07:25:13.687757 6653 server_base.cc:1034] running on GCE node
W20260925 07:25:13.687732 6659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:13.687748 6661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:13.687742 6658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:13.688068 6653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:25:13.688310 6653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:25:13.689460 6653 hybrid_clock.cc:648] HybridClock initialized: now 1790321113689441 us; error 30 us; skew 500 ppm
Sep 25 07:25:13 dist-test-slave-1drq krb5kdc[6278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321113, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:25:13.691767 6653 init.cc:377] Logged in from keytab as kudu/127.5.158.67@KRBTEST.COM (short username kudu)
I20260925 07:25:13.692154 6653 webserver.cc:460] Webserver started at http://127.5.158.67:33407/ using document root <none> and password file <none>
I20260925 07:25:13.692354 6653 fs_manager.cc:359] Metadata directory not provided
I20260925 07:25:13.692391 6653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:25:13.692512 6653 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:25:13.693391 6653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/ts-2/data/instance:
uuid: "cd4059c8f8124d25acc4181f455a180e"
format_stamp: "Formatted at 2026-09-25 07:25:13 on dist-test-slave-1drq"
server_key: "b093a417bb5526e2bc2437814b19554f"
server_key_iv: "5958cb8458edc9f7008bcf5e0519c6e7"
server_key_version: "encryptionkey@0"
I20260925 07:25:13.693683 6653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/ts-2/wal/instance:
uuid: "cd4059c8f8124d25acc4181f455a180e"
format_stamp: "Formatted at 2026-09-25 07:25:13 on dist-test-slave-1drq"
server_key: "b093a417bb5526e2bc2437814b19554f"
server_key_iv: "5958cb8458edc9f7008bcf5e0519c6e7"
server_key_version: "encryptionkey@0"
I20260925 07:25:13.694797 6653 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 07:25:13.695487 6668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:13.695657 6653 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:25:13.695720 6653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/ts-2/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/ts-2/wal
uuid: "cd4059c8f8124d25acc4181f455a180e"
format_stamp: "Formatted at 2026-09-25 07:25:13 on dist-test-slave-1drq"
server_key: "b093a417bb5526e2bc2437814b19554f"
server_key_iv: "5958cb8458edc9f7008bcf5e0519c6e7"
server_key_version: "encryptionkey@0"
I20260925 07:25:13.695806 6653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:25:13.702349 6653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:25:13.704363 6653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:25:13.704476 6653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:25:13.704675 6653 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:25:13.704955 6653 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:25:13.704988 6653 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:13.705006 6653 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:25:13.705051 6653 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:13.710418 6653 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.67:45217
I20260925 07:25:13.710484 6781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.67:45217 every 8 connection(s)
I20260925 07:25:13.710729 6653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
I20260925 07:25:13.712503 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 6653
I20260925 07:25:13.712632 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790321110323796-5753-0/minicluster-data/ts-2/wal/instance
I20260925 07:25:13.712785 5753 external_mini_cluster.cc:1428] Setting key 9ab98e3d917f0cc8960e1dab61337f65
Sep 25 07:25:13 dist-test-slave-1drq krb5kdc[6278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321113, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.67@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:25:13.718950 6382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:13.711313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:60837 (local address 127.5.158.126:41335)
0925 07:25:13.711365 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:13.711367 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:13.711516 (+ 149us) server_negotiation.cc:407] Connection header received
0925 07:25:13.712307 (+ 791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:13.712308 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:13.712317 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:13.712447 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:13.713159 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:13.713780 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:13.714612 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:13.714775 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:13.714820 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:13.716620 (+ 1800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:13.716621 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:13.716621 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:13.716627 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:13.717984 (+ 1357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:13.718276 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:13.718276 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:13.718277 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:13.718319 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:13.718507 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:13.718507 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:13.718508 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:13.718593 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:13.718638 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:13.718826 (+ 188us) server_negotiation.cc:299] Negotiation successful
0925 07:25:13.718878 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260925 07:25:13.719100 6784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:13.711281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:41335 (local address 127.5.158.67:60837)
0925 07:25:13.711420 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:13.711454 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:13.712227 (+ 773us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:13.712495 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:13.712497 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:13.712836 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:13.713072 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:13.713077 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:13.713850 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:13.713851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:13.714537 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:13.714548 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:13.714835 (+ 287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:13.714836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:13.714913 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:13.715276 (+ 363us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:13.715283 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:13.716548 (+ 1265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:13.718043 (+ 1495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:13.718047 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:13.718048 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:13.718218 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:13.718382 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:13.718382 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:13.718382 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:13.718459 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:13.718641 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:13.718645 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:13.718731 (+ 86us) client_negotiation.cc:769] Sending connection context
0925 07:25:13.718800 (+ 69us) client_negotiation.cc:240] Negotiation successful
0925 07:25:13.718891 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":79,"threads_started":1}
I20260925 07:25:13.719584 6782 heartbeater.cc:347] Connected to a master server at 127.5.158.126:41335
I20260925 07:25:13.719663 6782 heartbeater.cc:464] Registering TS with master...
I20260925 07:25:13.719806 6782 heartbeater.cc:511] Master 127.5.158.126:41335 requested a full tablet report, sending...
I20260925 07:25:13.720124 6322 ts_manager.cc:194] Registered new tserver with Master: cd4059c8f8124d25acc4181f455a180e (127.5.158.67:45217)
I20260925 07:25:13.720431 6322 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.67@KRBTEST.COM'} at 127.5.158.67:60837
I20260925 07:25:13.724160 5753 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 07:25:13.730902 6382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:13.725604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49740 (local address 127.5.158.126:41335)
0925 07:25:13.725675 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:13.725677 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:13.725771 (+ 94us) server_negotiation.cc:407] Connection header received
0925 07:25:13.725842 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:13.725843 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:13.725853 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:13.725908 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:13.726470 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:13.726870 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:13.727529 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:13.727698 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:13.727758 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:13.728902 (+ 1144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:13.728903 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:13.728903 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:13.728909 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:13.730089 (+ 1180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:13.730376 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:13.730377 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:13.730377 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:13.730418 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:13.730541 (+ 123us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:13.730541 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:13.730542 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:13.730641 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:13.730697 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:13.730768 (+ 71us) server_negotiation.cc:299] Negotiation successful
0925 07:25:13.730845 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 07:25:13.732486 6322 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49740:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 07:25:13.734036 6322 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 07:25:13.739740 6791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:13.735972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.67:45217 (local address 127.0.0.1:57558)
0925 07:25:13.736097 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:13.736131 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:13.736187 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:13.736597 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:13.736600 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:13.736602 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:25:13.736655 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:13.736656 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:13.737948 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:13.737949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:13.738747 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:13.738749 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:13.739481 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:13.739481 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:13.739555 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:13.739565 (+ 10us) client_negotiation.cc:769] Sending connection context
0925 07:25:13.739623 (+ 58us) client_negotiation.cc:240] Negotiation successful
0925 07:25:13.739664 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"mutex_wait_us":30,"thread_start_us":87,"threads_started":1}
I20260925 07:25:13.739974 6792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:13.736117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57558 (local address 127.5.158.67:45217)
0925 07:25:13.736454 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:13.736457 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:13.736465 (+ 8us) server_negotiation.cc:407] Connection header received
0925 07:25:13.736498 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:13.736499 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:13.736509 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:13.736590 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:25:13.736734 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:13.737842 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:13.738832 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:13.739339 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:13.739401 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:13.739448 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:13.739878 (+ 430us) server_negotiation.cc:299] Negotiation successful
0925 07:25:13.739912 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":70,"threads_started":1}
I20260925 07:25:13.740221 6790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:13.736005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.66:34615 (local address 127.0.0.1:42996)
0925 07:25:13.736146 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:13.736157 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:13.736209 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:13.736774 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:13.736774 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:13.736776 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:25:13.736813 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:13.736814 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:13.737890 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:13.737891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:13.739314 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:13.739316 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:13.740007 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:13.740008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:13.740066 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:13.740070 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 07:25:13.740123 (+ 53us) client_negotiation.cc:240] Negotiation successful
0925 07:25:13.740159 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":21,"thread_start_us":52,"threads_started":1}
I20260925 07:25:13.740232 6794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:13.736262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42996 (local address 127.5.158.66:34615)
0925 07:25:13.736642 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:13.736645 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:13.736654 (+ 9us) server_negotiation.cc:407] Connection header received
0925 07:25:13.736688 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:13.736690 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:13.736703 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:13.736775 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:25:13.736882 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:13.737779 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:13.739393 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:13.739892 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:13.740003 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:13.740056 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:13.740134 (+ 78us) server_negotiation.cc:299] Negotiation successful
0925 07:25:13.740168 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":98,"threads_started":1}
I20260925 07:25:13.740336 6793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:13.736192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:35467 (local address 127.0.0.1:37828)
0925 07:25:13.736330 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:13.736339 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:13.736375 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:13.736840 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:13.736861 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:13.736863 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:25:13.736934 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:13.736936 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:13.738176 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:13.738176 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:13.739141 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:13.739143 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:13.740114 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:13.740115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:13.740173 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:13.740179 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 07:25:13.740226 (+ 47us) client_negotiation.cc:240] Negotiation successful
0925 07:25:13.740267 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":85,"threads_started":1}
I20260925 07:25:13.740506 6795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:13.736441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37828 (local address 127.5.158.65:35467)
0925 07:25:13.736588 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:13.736593 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:13.736604 (+ 11us) server_negotiation.cc:407] Connection header received
0925 07:25:13.736640 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:13.736642 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:13.736656 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:13.736731 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:25:13.737023 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:13.738084 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:13.739217 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:13.739965 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:13.740028 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:13.740084 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:13.740393 (+ 309us) server_negotiation.cc:299] Negotiation successful
0925 07:25:13.740437 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":88,"threads_started":1}
I20260925 07:25:13.740792 6716 tablet_service.cc:1467] Processing CreateTablet for tablet 1c5b5b79045e4fc699ec45569ddbd804 (DEFAULT_TABLE table=test-table [id=7a9645e4023d4377aa77315700748976]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:25:13.741042 6716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c5b5b79045e4fc699ec45569ddbd804. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:25:13.743227 6796 tablet_bootstrap.cc:492] T 1c5b5b79045e4fc699ec45569ddbd804 P cd4059c8f8124d25acc4181f455a180e: Bootstrap starting.
I20260925 07:25:13.743621 6448 tablet_service.cc:1467] Processing CreateTablet for tablet 1c5b5b79045e4fc699ec45569ddbd804 (DEFAULT_TABLE table=test-table [id=7a9645e4023d4377aa77315700748976]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:25:13.743773 6796 tablet_bootstrap.cc:654] T 1c5b5b79045e4fc699ec45569ddbd804 P cd4059c8f8124d25acc4181f455a180e: Neither blocks nor log segments found. Creating new log.
I20260925 07:25:13.743868 6448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c5b5b79045e4fc699ec45569ddbd804. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:25:13.744375 6796 log.cc:824] T 1c5b5b79045e4fc699ec45569ddbd804 P cd4059c8f8124d25acc4181f455a180e: Log is configured to *not* fsync() on all Append() calls
I20260925 07:25:13.745026 6796 tablet_bootstrap.cc:492] T 1c5b5b79045e4fc699ec45569ddbd804 P cd4059c8f8124d25acc4181f455a180e: No bootstrap required, opened a new log
I20260925 07:25:13.745101 6796 ts_tablet_manager.cc:1397] T 1c5b5b79045e4fc699ec45569ddbd804 P cd4059c8f8124d25acc4181f455a180e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 07:25:13.745009 6582 tablet_service.cc:1467] Processing CreateTablet for tablet 1c5b5b79045e4fc699ec45569ddbd804 (DEFAULT_TABLE table=test-table [id=7a9645e4023d4377aa77315700748976]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:25:13.745368 6582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c5b5b79045e4fc699ec45569ddbd804. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:25:13.746469 6798 tablet_bootstrap.cc:492] T 1c5b5b79045e4fc699ec45569ddbd804 P f162dd4c688b4f8890da9e6f768f6f08: Bootstrap starting.
I20260925 07:25:13.747000 6798 tablet_bootstrap.cc:654] T 1c5b5b79045e4fc699ec45569ddbd804 P f162dd4c688b4f8890da9e6f768f6f08: Neither blocks nor log segments found. Creating new log.
I20260925 07:25:13.746981 6796 raft_consensus.cc:348] T 1c5b5b79045e4fc699ec45569ddbd804 P cd4059c8f8124d25acc4181f455a180e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd4059c8f8124d25acc4181f455a180e" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 45217 } } peers { permanent_uuid: "bdf6fab5ba1c4dc08f5a2520ac770951" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 34615 } } peers { permanent_uuid: "f162dd4c688b4f8890da9e6f768f6f08" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 35467 } }
I20260925 07:25:13.747143 6796 raft_consensus.cc:374] T 1c5b5b79045e4fc699ec45569ddbd804 P cd4059c8f8124d25acc4181f455a180e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:25:13.747181 6796 raft_consensus.cc:729] T 1c5b5b79045e4fc699ec45569ddbd804 P cd4059c8f8124d25acc4181f455a180e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd4059c8f8124d25acc4181f455a180e, State: Initialized, Role: FOLLOWER
I20260925 07:25:13.747295 6796 consensus_queue.cc:260] T 1c5b5b79045e4fc699ec45569ddbd804 P cd4059c8f8124d25acc4181f455a180e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd4059c8f8124d25acc4181f455a180e" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 45217 } } peers { permanent_uuid: "bdf6fab5ba1c4dc08f5a2520ac770951" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 34615 } } peers { permanent_uuid: "f162dd4c688b4f8890da9e6f768f6f08" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 35467 } }
I20260925 07:25:13.747532 6796 ts_tablet_manager.cc:1428] T 1c5b5b79045e4fc699ec45569ddbd804 P cd4059c8f8124d25acc4181f455a180e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 07:25:13.747640 6782 heartbeater.cc:503] Master 127.5.158.126:41335 was elected leader, sending a full tablet report...
I20260925 07:25:13.747649 6798 log.cc:824] T 1c5b5b79045e4fc699ec45569ddbd804 P f162dd4c688b4f8890da9e6f768f6f08: Log is configured to *not* fsync() on all Append() calls
I20260925 07:25:13.748309 6798 tablet_bootstrap.cc:492] T 1c5b5b79045e4fc699ec45569ddbd804 P f162dd4c688b4f8890da9e6f768f6f08: No bootstrap required, opened a new log
I20260925 07:25:13.748292 6801 tablet_bootstrap.cc:492] T 1c5b5b79045e4fc699ec45569ddbd804 P bdf6fab5ba1c4dc08f5a2520ac770951: Bootstrap starting.
I20260925 07:25:13.748363 6798 ts_tablet_manager.cc:1397] T 1c5b5b79045e4fc699ec45569ddbd804 P f162dd4c688b4f8890da9e6f768f6f08: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 07:25:13.748831 6801 tablet_bootstrap.cc:654] T 1c5b5b79045e4fc699ec45569ddbd804 P bdf6fab5ba1c4dc08f5a2520ac770951: Neither blocks nor log segments found. Creating new log.
I20260925 07:25:13.749274 6801 log.cc:824] T 1c5b5b79045e4fc699ec45569ddbd804 P bdf6fab5ba1c4dc08f5a2520ac770951: Log is configured to *not* fsync() on all Append() calls
I20260925 07:25:13.749866 6801 tablet_bootstrap.cc:492] T 1c5b5b79045e4fc699ec45569ddbd804 P bdf6fab5ba1c4dc08f5a2520ac770951: No bootstrap required, opened a new log
I20260925 07:25:13.749908 6801 ts_tablet_manager.cc:1397] T 1c5b5b79045e4fc699ec45569ddbd804 P bdf6fab5ba1c4dc08f5a2520ac770951: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 07:25:13.749934 6798 raft_consensus.cc:348] T 1c5b5b79045e4fc699ec45569ddbd804 P f162dd4c688b4f8890da9e6f768f6f08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd4059c8f8124d25acc4181f455a180e" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 45217 } } peers { permanent_uuid: "bdf6fab5ba1c4dc08f5a2520ac770951" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 34615 } } peers { permanent_uuid: "f162dd4c688b4f8890da9e6f768f6f08" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 35467 } }
I20260925 07:25:13.750087 6798 raft_consensus.cc:374] T 1c5b5b79045e4fc699ec45569ddbd804 P f162dd4c688b4f8890da9e6f768f6f08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:25:13.750128 6798 raft_consensus.cc:729] T 1c5b5b79045e4fc699ec45569ddbd804 P f162dd4c688b4f8890da9e6f768f6f08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f162dd4c688b4f8890da9e6f768f6f08, State: Initialized, Role: FOLLOWER
I20260925 07:25:13.750218 6798 consensus_queue.cc:260] T 1c5b5b79045e4fc699ec45569ddbd804 P f162dd4c688b4f8890da9e6f768f6f08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd4059c8f8124d25acc4181f455a180e" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 45217 } } peers { permanent_uuid: "bdf6fab5ba1c4dc08f5a2520ac770951" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 34615 } } peers { permanent_uuid: "f162dd4c688b4f8890da9e6f768f6f08" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 35467 } }
I20260925 07:25:13.750437 6798 ts_tablet_manager.cc:1428] T 1c5b5b79045e4fc699ec45569ddbd804 P f162dd4c688b4f8890da9e6f768f6f08: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 07:25:13.750468 6514 heartbeater.cc:503] Master 127.5.158.126:41335 was elected leader, sending a full tablet report...
I20260925 07:25:13.751593 6801 raft_consensus.cc:348] T 1c5b5b79045e4fc699ec45569ddbd804 P bdf6fab5ba1c4dc08f5a2520ac770951 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd4059c8f8124d25acc4181f455a180e" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 45217 } } peers { permanent_uuid: "bdf6fab5ba1c4dc08f5a2520ac770951" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 34615 } } peers { permanent_uuid: "f162dd4c688b4f8890da9e6f768f6f08" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 35467 } }
I20260925 07:25:13.751735 6801 raft_consensus.cc:374] T 1c5b5b79045e4fc699ec45569ddbd804 P bdf6fab5ba1c4dc08f5a2520ac770951 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:25:13.751770 6801 raft_consensus.cc:729] T 1c5b5b79045e4fc699ec45569ddbd804 P bdf6fab5ba1c4dc08f5a2520ac770951 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdf6fab5ba1c4dc08f5a2520ac770951, State: Initialized, Role: FOLLOWER
I20260925 07:25:13.751855 6801 consensus_queue.cc:260] T 1c5b5b79045e4fc699ec45569ddbd804 P bdf6fab5ba1c4dc08f5a2520ac770951 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd4059c8f8124d25acc4181f455a180e" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 45217 } } peers { permanent_uuid: "bdf6fab5ba1c4dc08f5a2520ac770951" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 34615 } } peers { permanent_uuid: "f162dd4c688b4f8890da9e6f768f6f08" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 35467 } }
I20260925 07:25:13.752043 6801 ts_tablet_manager.cc:1428] T 1c5b5b79045e4fc699ec45569ddbd804 P bdf6fab5ba1c4dc08f5a2520ac770951: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 07:25:13.752117 6648 heartbeater.cc:503] Master 127.5.158.126:41335 was elected leader, sending a full tablet report...
W20260925 07:25:13.827863 6649 tablet.cc:2369] T 1c5b5b79045e4fc699ec45569ddbd804 P bdf6fab5ba1c4dc08f5a2520ac770951: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 07:25:13.829264 6804 raft_consensus.cc:482] T 1c5b5b79045e4fc699ec45569ddbd804 P bdf6fab5ba1c4dc08f5a2520ac770951 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 07:25:13.829360 6804 raft_consensus.cc:504] T 1c5b5b79045e4fc699ec45569ddbd804 P bdf6fab5ba1c4dc08f5a2520ac770951 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd4059c8f8124d25acc4181f455a180e" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 45217 } } peers { permanent_uuid: "bdf6fab5ba1c4dc08f5a2520ac770951" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 34615 } } peers { permanent_uuid: "f162dd4c688b4f8890da9e6f768f6f08" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 35467 } }
I20260925 07:25:13.829707 6804 leader_election.cc:288] T 1c5b5b79045e4fc699ec45569ddbd804 P bdf6fab5ba1c4dc08f5a2520ac770951 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd4059c8f8124d25acc4181f455a180e (127.5.158.67:45217), f162dd4c688b4f8890da9e6f768f6f08 (127.5.158.65:35467)
I20260925 07:25:13.833173 6650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:13.829865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.67:45217 (local address 127.5.158.66:59193)
0925 07:25:13.829922 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:13.829934 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:13.829996 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:13.830146 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:13.830146 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:13.830164 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:25:13.830233 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:13.830234 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:13.831018 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:13.831019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:13.832218 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:13.832221 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:13.832914 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:13.832915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:13.832985 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:13.832990 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 07:25:13.833042 (+ 52us) client_negotiation.cc:240] Negotiation successful
0925 07:25:13.833105 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":56}
I20260925 07:25:13.833180 6795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:13.829907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:53739 (local address 127.5.158.65:35467)
0925 07:25:13.829950 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:13.829952 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:13.830083 (+ 131us) server_negotiation.cc:407] Connection header received
0925 07:25:13.830111 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:13.830112 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:13.830121 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:13.830170 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:25:13.830302 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:13.830948 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:13.832309 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:13.832833 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:13.832904 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:13.832911 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:13.833067 (+ 156us) server_negotiation.cc:299] Negotiation successful
0925 07:25:13.833120 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 07:25:13.833180 6792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:13.829893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:59193 (local address 127.5.158.67:45217)
0925 07:25:13.829940 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:13.829942 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:13.830007 (+ 65us) server_negotiation.cc:407] Connection header received
0925 07:25:13.830058 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:13.830059 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:13.830071 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:13.830131 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:25:13.830307 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:13.830948 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:13.832307 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:13.832833 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:13.832903 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:13.832911 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:13.833065 (+ 154us) server_negotiation.cc:299] Negotiation successful
0925 07:25:13.833116 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260925 07:25:13.833179 6805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:13.829867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:35467 (local address 127.5.158.66:53739)
0925 07:25:13.830007 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:13.830015 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:13.830065 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:13.830182 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:13.830182 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:13.830183 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:25:13.830239 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:13.830240 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:13.831018 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:13.831019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:13.832218 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:13.832221 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:13.832914 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:13.832915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:13.832984 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:13.832991 (+ 7us) client_negotiation.cc:769] Sending connection context
0925 07:25:13.833042 (+ 51us) client_negotiation.cc:240] Negotiation successful
0925 07:25:13.833098 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":62,"thread_start_us":73,"threads_started":1}
I20260925 07:25:13.833588 6468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5b5b79045e4fc699ec45569ddbd804" candidate_uuid: "bdf6fab5ba1c4dc08f5a2520ac770951" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f162dd4c688b4f8890da9e6f768f6f08" is_pre_election: true
I20260925 07:25:13.833673 6736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5b5b79045e4fc699ec45569ddbd804" candidate_uuid: "bdf6fab5ba1c4dc08f5a2520ac770951" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd4059c8f8124d25acc4181f455a180e" is_pre_election: true
I20260925 07:25:13.833727 6468 raft_consensus.cc:2454] T 1c5b5b79045e4fc699ec45569ddbd804 P f162dd4c688b4f8890da9e6f768f6f08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdf6fab5ba1c4dc08f5a2520ac770951 in term 0.
I20260925 07:25:13.833777 6736 raft_consensus.cc:2454] T 1c5b5b79045e4fc699ec45569ddbd804 P cd4059c8f8124d25acc4181f455a180e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdf6fab5ba1c4dc08f5a2520ac770951 in term 0.
I20260925 07:25:13.833956 6537 leader_election.cc:302] T 1c5b5b79045e4fc699ec45569ddbd804 P bdf6fab5ba1c4dc08f5a2520ac770951 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdf6fab5ba1c4dc08f5a2520ac770951, f162dd4c688b4f8890da9e6f768f6f08; no voters:
I20260925 07:25:13.834110 6804 raft_consensus.cc:2781] T 1c5b5b79045e4fc699ec45569ddbd804 P bdf6fab5ba1c4dc08f5a2520ac770951 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 07:25:13.834143 6804 raft_consensus.cc:482] T 1c5b5b79045e4fc699ec45569ddbd804 P bdf6fab5ba1c4dc08f5a2520ac770951 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 07:25:13.834156 6804 raft_consensus.cc:3037] T 1c5b5b79045e4fc699ec45569ddbd804 P bdf6fab5ba1c4dc08f5a2520ac770951 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:25:13.834731 6804 raft_consensus.cc:504] T 1c5b5b79045e4fc699ec45569ddbd804 P bdf6fab5ba1c4dc08f5a2520ac770951 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd4059c8f8124d25acc4181f455a180e" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 45217 } } peers { permanent_uuid: "bdf6fab5ba1c4dc08f5a2520ac770951" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 34615 } } peers { permanent_uuid: "f162dd4c688b4f8890da9e6f768f6f08" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 35467 } }
I20260925 07:25:13.834848 6804 leader_election.cc:288] T 1c5b5b79045e4fc699ec45569ddbd804 P bdf6fab5ba1c4dc08f5a2520ac770951 [CANDIDATE]: Term 1 election: Requested vote from peers cd4059c8f8124d25acc4181f455a180e (127.5.158.67:45217), f162dd4c688b4f8890da9e6f768f6f08 (127.5.158.65:35467)
I20260925 07:25:13.835013 6736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5b5b79045e4fc699ec45569ddbd804" candidate_uuid: "bdf6fab5ba1c4dc08f5a2520ac770951" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd4059c8f8124d25acc4181f455a180e"
I20260925 07:25:13.835032 6468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c5b5b79045e4fc699ec45569ddbd804" candidate_uuid: "bdf6fab5ba1c4dc08f5a2520ac770951" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f162dd4c688b4f8890da9e6f768f6f08"
I20260925 07:25:13.835081 6736 raft_consensus.cc:3037] T 1c5b5b79045e4fc699ec45569ddbd804 P cd4059c8f8124d25acc4181f455a180e [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:25:13.835150 6468 raft_consensus.cc:3037] T 1c5b5b79045e4fc699ec45569ddbd804 P f162dd4c688b4f8890da9e6f768f6f08 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:25:13.835794 6736 raft_consensus.cc:2454] T 1c5b5b79045e4fc699ec45569ddbd804 P cd4059c8f8124d25acc4181f455a180e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdf6fab5ba1c4dc08f5a2520ac770951 in term 1.
I20260925 07:25:13.835858 6468 raft_consensus.cc:2454] T 1c5b5b79045e4fc699ec45569ddbd804 P f162dd4c688b4f8890da9e6f768f6f08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdf6fab5ba1c4dc08f5a2520ac770951 in term 1.
I20260925 07:25:13.835974 6536 leader_election.cc:302] T 1c5b5b79045e4fc699ec45569ddbd804 P bdf6fab5ba1c4dc08f5a2520ac770951 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdf6fab5ba1c4dc08f5a2520ac770951, cd4059c8f8124d25acc4181f455a180e; no voters:
I20260925 07:25:13.836097 6804 raft_consensus.cc:2781] T 1c5b5b79045e4fc699ec45569ddbd804 P bdf6fab5ba1c4dc08f5a2520ac770951 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:25:13.836252 6804 raft_consensus.cc:686] T 1c5b5b79045e4fc699ec45569ddbd804 P bdf6fab5ba1c4dc08f5a2520ac770951 [term 1 LEADER]: Becoming Leader. State: Replica: bdf6fab5ba1c4dc08f5a2520ac770951, State: Running, Role: LEADER
I20260925 07:25:13.836372 6804 consensus_queue.cc:237] T 1c5b5b79045e4fc699ec45569ddbd804 P bdf6fab5ba1c4dc08f5a2520ac770951 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd4059c8f8124d25acc4181f455a180e" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 45217 } } peers { permanent_uuid: "bdf6fab5ba1c4dc08f5a2520ac770951" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 34615 } } peers { permanent_uuid: "f162dd4c688b4f8890da9e6f768f6f08" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 35467 } }
I20260925 07:25:13.837133 6322 catalog_manager.cc:5495] T 1c5b5b79045e4fc699ec45569ddbd804 P bdf6fab5ba1c4dc08f5a2520ac770951 reported cstate change: term changed from 0 to 1, leader changed from <none> to bdf6fab5ba1c4dc08f5a2520ac770951 (127.5.158.66). New cstate: current_term: 1 leader_uuid: "bdf6fab5ba1c4dc08f5a2520ac770951" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd4059c8f8124d25acc4181f455a180e" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 45217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdf6fab5ba1c4dc08f5a2520ac770951" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 34615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f162dd4c688b4f8890da9e6f768f6f08" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 35467 } health_report { overall_health: UNKNOWN } } }
Sep 25 07:25:13 dist-test-slave-1drq krb5kdc[6278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321113, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.158.65@KRBTEST.COM
I20260925 07:25:13.856793 6795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:13.850371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37844 (local address 127.5.158.65:35467)
0925 07:25:13.850402 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:13.850404 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:13.850494 (+ 90us) server_negotiation.cc:407] Connection header received
0925 07:25:13.850537 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:13.850537 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:13.850543 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:13.850591 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:13.851015 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:13.851425 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:13.852024 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:13.852175 (+ 151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:13.852219 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:13.854341 (+ 2122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:13.854347 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:13.854347 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:13.854353 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:13.855911 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:13.856194 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:13.856196 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:13.856196 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:13.856248 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:13.856431 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:13.856431 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:13.856432 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:13.856528 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:13.856624 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:13.856691 (+ 67us) server_negotiation.cc:299] Negotiation successful
0925 07:25:13.856744 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
Sep 25 07:25:13 dist-test-slave-1drq krb5kdc[6278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321113, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 07:25:13 dist-test-slave-1drq krb5kdc[6278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321113, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.158.65@KRBTEST.COM
I20260925 07:25:13.876359 6795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:13.870720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37852 (local address 127.5.158.65:35467)
0925 07:25:13.870770 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:13.870772 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:13.870920 (+ 148us) server_negotiation.cc:407] Connection header received
0925 07:25:13.870968 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:13.870968 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:13.870975 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:13.871016 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:13.871477 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:13.871863 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:13.872447 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:13.872604 (+ 157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:13.872645 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:13.874441 (+ 1796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:13.874442 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:13.874442 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:13.874448 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:13.875548 (+ 1100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:13.875831 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:13.875832 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:13.875832 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:13.875877 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:13.876017 (+ 140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:13.876017 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:13.876017 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:13.876089 (+ 72us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:13.876149 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:13.876256 (+ 107us) server_negotiation.cc:299] Negotiation successful
0925 07:25:13.876306 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
Sep 25 07:25:13 dist-test-slave-1drq krb5kdc[6278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321113, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 07:25:13 dist-test-slave-1drq krb5kdc[6278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321113, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.158.65@KRBTEST.COM
I20260925 07:25:13.895140 6795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:13.889142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37858 (local address 127.5.158.65:35467)
0925 07:25:13.889180 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:13.889181 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:13.889283 (+ 102us) server_negotiation.cc:407] Connection header received
0925 07:25:13.889352 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:13.889353 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:13.889362 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:13.889455 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:13.889978 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:13.890425 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:13.891153 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:13.891329 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:13.891404 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:13.893265 (+ 1861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:13.893266 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:13.893266 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:13.893271 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:13.894337 (+ 1066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:13.894622 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:13.894622 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:13.894623 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:13.894667 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:13.894808 (+ 141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:13.894808 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:13.894809 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:13.894879 (+ 70us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:13.894940 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:13.895026 (+ 86us) server_negotiation.cc:299] Negotiation successful
0925 07:25:13.895073 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260925 07:25:13.895610 6428 tablet_service.cc:2908] Rejecting scan request for tablet 1c5b5b79045e4fc699ec45569ddbd804: Uninitialized: clean time has not yet been initialized
I20260925 07:25:13.896459 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 6385
I20260925 07:25:13.900810 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 6519
I20260925 07:25:13.905303 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 6653
I20260925 07:25:13.909617 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 6291
2026-09-25T07:25:13Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (1801 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 07:25:13 dist-test-slave-1drq krb5kdc[6820](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 07:25:13 dist-test-slave-1drq krb5kdc[6820](info): set up 2 sockets
Sep 25 07:25:13 dist-test-slave-1drq krb5kdc[6820](info): commencing operation
krb5kdc: starting...
W20260925 07:25:14.864197 5753 mini_kdc.cc:121] Time spent starting KDC: real 0.935s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Sep 25 07:25:14 dist-test-slave-1drq krb5kdc[6820](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321114, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T07:25:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T07:25:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
I20260925 07:25:14.963310 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:39083
--webserver_interface=127.5.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:43541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.158.126:39083
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:25:15.035523 6833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:25:15.035704 6833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:25:15.035727 6833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:25:15.036839 6833 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:25:15.036876 6833 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260925 07:25:15.036888 6833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:25:15.036901 6833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:25:15.036913 6833 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:25:15.036926 6833 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:25:15.038290 6833 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:43541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.158.126:39083
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:39083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6833
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:25:15.038456 6833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:25:15.038600 6833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:25:15.042958 6838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:15.042961 6841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:15.043097 6833 server_base.cc:1034] running on GCE node
W20260925 07:25:15.042924 6839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:15.043429 6833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:25:15.043694 6833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:25:15.044874 6833 hybrid_clock.cc:648] HybridClock initialized: now 1790321115044859 us; error 41 us; skew 500 ppm
Sep 25 07:25:15 dist-test-slave-1drq krb5kdc[6820](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321115, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:25:15.047326 6833 init.cc:377] Logged in from keytab as kudu/127.5.158.126@KRBTEST.COM (short username kudu)
I20260925 07:25:15.047747 6833 webserver.cc:460] Webserver started at http://127.5.158.126:39895/ using document root <none> and password file <none>
I20260925 07:25:15.047955 6833 fs_manager.cc:359] Metadata directory not provided
I20260925 07:25:15.048025 6833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:25:15.048166 6833 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:25:15.049065 6833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/master-0/data/instance:
uuid: "e959c62d37d34aecaf9bf9e28cba7df9"
format_stamp: "Formatted at 2026-09-25 07:25:15 on dist-test-slave-1drq"
server_key: "1fd5880e5f5b19d3f38c0e25709da4bf"
server_key_iv: "9e0b831de6002d2d2d687d570e84f896"
server_key_version: "encryptionkey@0"
I20260925 07:25:15.049469 6833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/master-0/wal/instance:
uuid: "e959c62d37d34aecaf9bf9e28cba7df9"
format_stamp: "Formatted at 2026-09-25 07:25:15 on dist-test-slave-1drq"
server_key: "1fd5880e5f5b19d3f38c0e25709da4bf"
server_key_iv: "9e0b831de6002d2d2d687d570e84f896"
server_key_version: "encryptionkey@0"
I20260925 07:25:15.050705 6833 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 07:25:15.051409 6848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:15.051584 6833 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:25:15.051674 6833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "e959c62d37d34aecaf9bf9e28cba7df9"
format_stamp: "Formatted at 2026-09-25 07:25:15 on dist-test-slave-1drq"
server_key: "1fd5880e5f5b19d3f38c0e25709da4bf"
server_key_iv: "9e0b831de6002d2d2d687d570e84f896"
server_key_version: "encryptionkey@0"
I20260925 07:25:15.051753 6833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:25:15.083119 6833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:25:15.085072 6833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:25:15.085230 6833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:25:15.089618 6833 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.126:39083
I20260925 07:25:15.089648 6910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.126:39083 every 8 connection(s)
I20260925 07:25:15.089942 6833 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:25:15.090489 6911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:25:15.092453 6911 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e959c62d37d34aecaf9bf9e28cba7df9: Bootstrap starting.
I20260925 07:25:15.092964 6911 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e959c62d37d34aecaf9bf9e28cba7df9: Neither blocks nor log segments found. Creating new log.
I20260925 07:25:15.093276 6911 log.cc:824] T 00000000000000000000000000000000 P e959c62d37d34aecaf9bf9e28cba7df9: Log is configured to *not* fsync() on all Append() calls
I20260925 07:25:15.093847 6911 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e959c62d37d34aecaf9bf9e28cba7df9: No bootstrap required, opened a new log
I20260925 07:25:15.095768 6911 raft_consensus.cc:348] T 00000000000000000000000000000000 P e959c62d37d34aecaf9bf9e28cba7df9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e959c62d37d34aecaf9bf9e28cba7df9" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 39083 } }
I20260925 07:25:15.095916 6911 raft_consensus.cc:374] T 00000000000000000000000000000000 P e959c62d37d34aecaf9bf9e28cba7df9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:25:15.095961 6911 raft_consensus.cc:729] T 00000000000000000000000000000000 P e959c62d37d34aecaf9bf9e28cba7df9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e959c62d37d34aecaf9bf9e28cba7df9, State: Initialized, Role: FOLLOWER
I20260925 07:25:15.096117 6911 consensus_queue.cc:260] T 00000000000000000000000000000000 P e959c62d37d34aecaf9bf9e28cba7df9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e959c62d37d34aecaf9bf9e28cba7df9" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 39083 } }
I20260925 07:25:15.096210 6911 raft_consensus.cc:388] T 00000000000000000000000000000000 P e959c62d37d34aecaf9bf9e28cba7df9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 07:25:15.096254 6911 raft_consensus.cc:482] T 00000000000000000000000000000000 P e959c62d37d34aecaf9bf9e28cba7df9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 07:25:15.096302 6911 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e959c62d37d34aecaf9bf9e28cba7df9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:25:15.096796 6911 raft_consensus.cc:504] T 00000000000000000000000000000000 P e959c62d37d34aecaf9bf9e28cba7df9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e959c62d37d34aecaf9bf9e28cba7df9" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 39083 } }
I20260925 07:25:15.096900 6911 leader_election.cc:302] T 00000000000000000000000000000000 P e959c62d37d34aecaf9bf9e28cba7df9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e959c62d37d34aecaf9bf9e28cba7df9; no voters:
I20260925 07:25:15.097056 6911 leader_election.cc:288] T 00000000000000000000000000000000 P e959c62d37d34aecaf9bf9e28cba7df9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 07:25:15.097311 6911 sys_catalog.cc:564] T 00000000000000000000000000000000 P e959c62d37d34aecaf9bf9e28cba7df9 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:25:15.097349 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 6833
I20260925 07:25:15.097435 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/master-0/wal/instance
I20260925 07:25:15.097600 5753 external_mini_cluster.cc:1428] Setting key 35ffa224757133f9d9a6240f5ab78e95
I20260925 07:25:15.097765 6914 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e959c62d37d34aecaf9bf9e28cba7df9 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:25:15.097918 6914 raft_consensus.cc:686] T 00000000000000000000000000000000 P e959c62d37d34aecaf9bf9e28cba7df9 [term 1 LEADER]: Becoming Leader. State: Replica: e959c62d37d34aecaf9bf9e28cba7df9, State: Running, Role: LEADER
I20260925 07:25:15.097985 6914 consensus_queue.cc:237] T 00000000000000000000000000000000 P e959c62d37d34aecaf9bf9e28cba7df9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e959c62d37d34aecaf9bf9e28cba7df9" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 39083 } }
I20260925 07:25:15.098291 6914 sys_catalog.cc:455] T 00000000000000000000000000000000 P e959c62d37d34aecaf9bf9e28cba7df9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e959c62d37d34aecaf9bf9e28cba7df9. Latest consensus state: current_term: 1 leader_uuid: "e959c62d37d34aecaf9bf9e28cba7df9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e959c62d37d34aecaf9bf9e28cba7df9" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 39083 } } }
I20260925 07:25:15.098359 6914 sys_catalog.cc:458] T 00000000000000000000000000000000 P e959c62d37d34aecaf9bf9e28cba7df9 [sys.catalog]: This master's current role is: LEADER
I20260925 07:25:15.098538 6915 sys_catalog.cc:455] T 00000000000000000000000000000000 P e959c62d37d34aecaf9bf9e28cba7df9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e959c62d37d34aecaf9bf9e28cba7df9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e959c62d37d34aecaf9bf9e28cba7df9" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 39083 } } }
I20260925 07:25:15.098727 6915 sys_catalog.cc:458] T 00000000000000000000000000000000 P e959c62d37d34aecaf9bf9e28cba7df9 [sys.catalog]: This master's current role is: LEADER
I20260925 07:25:15.098533 6931 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 07:25:15.099118 6931 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 07:25:15.101202 6931 catalog_manager.cc:1347] Generated new cluster ID: 07054b23ac4245c9954a1a06e1df917e
I20260925 07:25:15.101251 6931 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 07:25:15 dist-test-slave-1drq krb5kdc[6820](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321114, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:25:15.107467 6924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:15.098147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39350 (local address 127.5.158.126:39083)
0925 07:25:15.098643 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:15.098650 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:15.098676 (+ 26us) server_negotiation.cc:407] Connection header received
0925 07:25:15.098770 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:15.098775 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:15.098820 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:15.098936 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:15.099476 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.100727 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.101565 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.101947 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.102092 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:15.104064 (+ 1972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:15.104068 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:15.104068 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:15.104078 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:15.106075 (+ 1997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:15.106449 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:15.106452 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:15.106452 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:15.106533 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:15.106761 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:15.106761 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:15.106762 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:15.106948 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:15.107091 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:15.107164 (+ 73us) server_negotiation.cc:299] Negotiation successful
0925 07:25:15.107296 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":71,"threads_started":1}
I20260925 07:25:15.112856 6931 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 07:25:15.113317 6931 catalog_manager.cc:1504] Loading token signing keys...
I20260925 07:25:15.121519 6931 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e959c62d37d34aecaf9bf9e28cba7df9: Generated new TSK 0
I20260925 07:25:15.121670 6931 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.158.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab.
Entry for principal kudu/127.5.158.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab.
I20260925 07:25:15.147028 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.65:0
--local_ip_for_outbound_sockets=127.5.158.65
--webserver_interface=127.5.158.65
--webserver_port=0
--tserver_master_addrs=127.5.158.126:39083
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab
--principal=kudu/127.5.158.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:43541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:25:15.228638 6937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:25:15.228789 6937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:25:15.228808 6937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:25:15.229904 6937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:25:15.229939 6937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:25:15.229957 6937 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 07:25:15.229972 6937 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.65
I20260925 07:25:15.231246 6937 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:43541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab
--principal=kudu/127.5.158.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.158.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.158.126:39083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6937
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.65
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:25:15.231434 6937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:25:15.231637 6937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:25:15.236037 6945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:15.236028 6943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:15.236017 6937 server_base.cc:1034] running on GCE node
W20260925 07:25:15.236012 6942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:15.236440 6937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:25:15.236712 6937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:25:15.237865 6937 hybrid_clock.cc:648] HybridClock initialized: now 1790321115237855 us; error 26 us; skew 500 ppm
Sep 25 07:25:15 dist-test-slave-1drq krb5kdc[6820](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321115, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:25:15.239926 6937 init.cc:377] Logged in from keytab as kudu/127.5.158.65@KRBTEST.COM (short username kudu)
I20260925 07:25:15.240285 6937 webserver.cc:460] Webserver started at http://127.5.158.65:46875/ using document root <none> and password file <none>
I20260925 07:25:15.240485 6937 fs_manager.cc:359] Metadata directory not provided
I20260925 07:25:15.240562 6937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:25:15.240698 6937 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:25:15.241653 6937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/ts-0/data/instance:
uuid: "74cbfed8f72549e0846b0e9636ad002c"
format_stamp: "Formatted at 2026-09-25 07:25:15 on dist-test-slave-1drq"
server_key: "bfb67beb6448fbd060ee4bd38c696127"
server_key_iv: "5fbe249d77ae3d4461192b58e19b038a"
server_key_version: "encryptionkey@0"
I20260925 07:25:15.242000 6937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/ts-0/wal/instance:
uuid: "74cbfed8f72549e0846b0e9636ad002c"
format_stamp: "Formatted at 2026-09-25 07:25:15 on dist-test-slave-1drq"
server_key: "bfb67beb6448fbd060ee4bd38c696127"
server_key_iv: "5fbe249d77ae3d4461192b58e19b038a"
server_key_version: "encryptionkey@0"
I20260925 07:25:15.243182 6937 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 07:25:15.243777 6952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:15.243913 6937 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:25:15.243995 6937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/ts-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/ts-0/wal
uuid: "74cbfed8f72549e0846b0e9636ad002c"
format_stamp: "Formatted at 2026-09-25 07:25:15 on dist-test-slave-1drq"
server_key: "bfb67beb6448fbd060ee4bd38c696127"
server_key_iv: "5fbe249d77ae3d4461192b58e19b038a"
server_key_version: "encryptionkey@0"
I20260925 07:25:15.244065 6937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:25:15.255424 6937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:25:15.257174 6937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:25:15.257311 6937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:25:15.257727 6937 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:25:15.257783 6937 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:15.257839 6937 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:25:15.257866 6937 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:15.263680 6937 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.65:36607
I20260925 07:25:15.263728 7067 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.65:36607 every 8 connection(s)
Sep 25 07:25:15 dist-test-slave-1drq krb5kdc[6820](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321115, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.65@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:25:15.263975 6937 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
I20260925 07:25:15.266753 6924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:15.258194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:53323 (local address 127.5.158.126:39083)
0925 07:25:15.258258 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:15.258260 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:15.258418 (+ 158us) server_negotiation.cc:407] Connection header received
0925 07:25:15.258838 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:15.258839 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:15.258851 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:15.258904 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:15.259565 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.260265 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.261763 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.262052 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.262103 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:15.264509 (+ 2406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:15.264510 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:15.264510 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:15.264517 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:15.265694 (+ 1177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:15.266039 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:15.266040 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:15.266040 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:15.266088 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:15.266295 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:15.266296 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:15.266296 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:15.266387 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:15.266434 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:15.266622 (+ 188us) server_negotiation.cc:299] Negotiation successful
0925 07:25:15.266701 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260925 07:25:15.266943 6961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:15.258178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:39083 (local address 127.5.158.65:53323)
0925 07:25:15.258322 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:15.258354 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:15.258735 (+ 381us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:15.258957 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:15.258961 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:15.259257 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:15.259479 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:15.259484 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.260364 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.260365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:15.261669 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:15.261672 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.262140 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.262141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:15.262281 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:15.262765 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:15.262774 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:15.264407 (+ 1633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:15.265762 (+ 1355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:15.265767 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:15.265769 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:15.265968 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:15.266144 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:15.266145 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:15.266145 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:15.266242 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:15.266438 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:15.266442 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:15.266538 (+ 96us) client_negotiation.cc:769] Sending connection context
0925 07:25:15.266611 (+ 73us) client_negotiation.cc:240] Negotiation successful
0925 07:25:15.266733 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":90,"threads_started":1}
I20260925 07:25:15.267355 7068 heartbeater.cc:347] Connected to a master server at 127.5.158.126:39083
I20260925 07:25:15.267428 7068 heartbeater.cc:464] Registering TS with master...
I20260925 07:25:15.267602 7068 heartbeater.cc:511] Master 127.5.158.126:39083 requested a full tablet report, sending...
I20260925 07:25:15.268040 6864 ts_manager.cc:194] Registered new tserver with Master: 74cbfed8f72549e0846b0e9636ad002c (127.5.158.65:36607)
I20260925 07:25:15.268877 6864 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.65@KRBTEST.COM'} at 127.5.158.65:53323
I20260925 07:25:15.271064 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 6937
I20260925 07:25:15.271157 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/ts-0/wal/instance
I20260925 07:25:15.271325 5753 external_mini_cluster.cc:1428] Setting key 959c51c14e62d1fa4ac461f9a6434b0d
I20260925 07:25:15.273280 6924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:15.268316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:38579 (local address 127.5.158.126:39083)
0925 07:25:15.268353 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:15.268354 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:15.268408 (+ 54us) server_negotiation.cc:407] Connection header received
0925 07:25:15.268459 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:15.268460 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:15.268467 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:15.268501 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:15.268792 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.269359 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.270025 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.270204 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.270254 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:15.271047 (+ 793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:15.271048 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:15.271048 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:15.271052 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:15.272339 (+ 1287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:15.272682 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:15.272682 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:15.272682 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:15.272723 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:15.272895 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:15.272896 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:15.272896 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:15.272986 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:15.273028 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:15.273184 (+ 156us) server_negotiation.cc:299] Negotiation successful
0925 07:25:15.273234 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 07:25:15.273382 7074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:15.268232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:39083 (local address 127.5.158.65:38579)
0925 07:25:15.268346 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:15.268356 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:15.268411 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:15.268517 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:15.268517 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:15.268685 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:15.268738 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:15.268739 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.269444 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.269444 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:15.269957 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:15.269958 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.270265 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.270265 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:15.270312 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:15.270685 (+ 373us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:15.270689 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:15.270972 (+ 283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:15.272405 (+ 1433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:15.272406 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:15.272407 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:15.272616 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:15.272777 (+ 161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:15.272777 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:15.272778 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:15.272834 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:15.273041 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:15.273042 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:15.273098 (+ 56us) client_negotiation.cc:769] Sending connection context
0925 07:25:15.273163 (+ 65us) client_negotiation.cc:240] Negotiation successful
0925 07:25:15.273256 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":72,"threads_started":1}
WARNING: no policy specified for kudu/127.5.158.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab.
Entry for principal kudu/127.5.158.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab.
I20260925 07:25:15.296450 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.66:0
--local_ip_for_outbound_sockets=127.5.158.66
--webserver_interface=127.5.158.66
--webserver_port=0
--tserver_master_addrs=127.5.158.126:39083
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab
--principal=kudu/127.5.158.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:43541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:25:15.368250 7077 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:25:15.368448 7077 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:25:15.368479 7077 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:25:15.369513 7077 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:25:15.369565 7077 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:25:15.369596 7077 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 07:25:15.369630 7077 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.66
I20260925 07:25:15.370909 7077 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:43541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab
--principal=kudu/127.5.158.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.158.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.158.126:39083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7077
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.66
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:25:15.371140 7077 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:25:15.371332 7077 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:25:15.376370 7082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:15.376421 7083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:15.376423 7085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:15.376704 7077 server_base.cc:1034] running on GCE node
I20260925 07:25:15.376879 7077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:25:15.377112 7077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:25:15.378259 7077 hybrid_clock.cc:648] HybridClock initialized: now 1790321115378244 us; error 31 us; skew 500 ppm
Sep 25 07:25:15 dist-test-slave-1drq krb5kdc[6820](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321115, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:25:15.380420 7077 init.cc:377] Logged in from keytab as kudu/127.5.158.66@KRBTEST.COM (short username kudu)
I20260925 07:25:15.380786 7077 webserver.cc:460] Webserver started at http://127.5.158.66:35221/ using document root <none> and password file <none>
I20260925 07:25:15.380986 7077 fs_manager.cc:359] Metadata directory not provided
I20260925 07:25:15.381026 7077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:25:15.381197 7077 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:25:15.382067 7077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/ts-1/data/instance:
uuid: "7bb91ac7fc154b169ae6c1b68f9b00d4"
format_stamp: "Formatted at 2026-09-25 07:25:15 on dist-test-slave-1drq"
server_key: "79d9f217ab9af270af960564bf2e8b70"
server_key_iv: "fc024a982abd39f5d8623cb2ce8d8f5f"
server_key_version: "encryptionkey@0"
I20260925 07:25:15.382376 7077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/ts-1/wal/instance:
uuid: "7bb91ac7fc154b169ae6c1b68f9b00d4"
format_stamp: "Formatted at 2026-09-25 07:25:15 on dist-test-slave-1drq"
server_key: "79d9f217ab9af270af960564bf2e8b70"
server_key_iv: "fc024a982abd39f5d8623cb2ce8d8f5f"
server_key_version: "encryptionkey@0"
I20260925 07:25:15.383507 7077 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 07:25:15.384107 7092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:15.384271 7077 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:25:15.384348 7077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/ts-1/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/ts-1/wal
uuid: "7bb91ac7fc154b169ae6c1b68f9b00d4"
format_stamp: "Formatted at 2026-09-25 07:25:15 on dist-test-slave-1drq"
server_key: "79d9f217ab9af270af960564bf2e8b70"
server_key_iv: "fc024a982abd39f5d8623cb2ce8d8f5f"
server_key_version: "encryptionkey@0"
I20260925 07:25:15.384411 7077 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:25:15.397830 7077 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:25:15.399461 7077 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:25:15.399603 7077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:25:15.400024 7077 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:25:15.400063 7077 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:15.400080 7077 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:25:15.400112 7077 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:15.406339 7077 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.66:40785
I20260925 07:25:15.406411 7207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.66:40785 every 8 connection(s)
Sep 25 07:25:15 dist-test-slave-1drq krb5kdc[6820](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321115, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.66@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:25:15.406705 7077 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
I20260925 07:25:15.409276 6924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:15.400442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:49103 (local address 127.5.158.126:39083)
0925 07:25:15.400519 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:15.400522 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:15.400657 (+ 135us) server_negotiation.cc:407] Connection header received
0925 07:25:15.401031 (+ 374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:15.401032 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:15.401043 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:15.401125 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:15.401881 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.402633 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.404105 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.404403 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.404472 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:15.407014 (+ 2542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:15.407015 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:15.407015 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:15.407023 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:15.408038 (+ 1015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:15.408403 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:15.408405 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:15.408405 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:15.408460 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:15.408686 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:15.408687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:15.408688 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:15.408799 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:15.408870 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:15.409122 (+ 252us) server_negotiation.cc:299] Negotiation successful
0925 07:25:15.409195 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260925 07:25:15.409439 7101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:15.400416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:39083 (local address 127.5.158.66:49103)
0925 07:25:15.400543 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:15.400578 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:15.400947 (+ 369us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:15.401208 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:15.401211 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:15.401525 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:15.401781 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:15.401786 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.402741 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.402742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:15.404001 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:15.404004 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.404482 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.404483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:15.404599 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:15.405099 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:15.405109 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:15.406940 (+ 1831us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:15.408114 (+ 1174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:15.408120 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:15.408122 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:15.408319 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:15.408523 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:15.408524 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:15.408524 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:15.408625 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:15.408862 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:15.408867 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:15.408987 (+ 120us) client_negotiation.cc:769] Sending connection context
0925 07:25:15.409082 (+ 95us) client_negotiation.cc:240] Negotiation successful
0925 07:25:15.409248 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":70,"threads_started":1}
I20260925 07:25:15.409878 7208 heartbeater.cc:347] Connected to a master server at 127.5.158.126:39083
I20260925 07:25:15.409953 7208 heartbeater.cc:464] Registering TS with master...
I20260925 07:25:15.410108 7208 heartbeater.cc:511] Master 127.5.158.126:39083 requested a full tablet report, sending...
I20260925 07:25:15.410463 6864 ts_manager.cc:194] Registered new tserver with Master: 7bb91ac7fc154b169ae6c1b68f9b00d4 (127.5.158.66:40785)
I20260925 07:25:15.410509 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 7077
I20260925 07:25:15.410598 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/ts-1/wal/instance
I20260925 07:25:15.410758 5753 external_mini_cluster.cc:1428] Setting key 53f3d83d81b0d85a85bc2f4e9504a15a
I20260925 07:25:15.410897 6864 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.66@KRBTEST.COM'} at 127.5.158.66:49103
I20260925 07:25:15.416297 6924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:15.411053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:33231 (local address 127.5.158.126:39083)
0925 07:25:15.411309 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:15.411311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:15.411317 (+ 6us) server_negotiation.cc:407] Connection header received
0925 07:25:15.411331 (+ 14us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:15.411349 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:15.411356 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:15.411407 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:15.411709 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.412161 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.412763 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.412936 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.413006 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:15.413872 (+ 866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:15.413872 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:15.413873 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:15.413878 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:15.415186 (+ 1308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:15.415614 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:15.415615 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:15.415615 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:15.415661 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:15.415860 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:15.415860 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:15.415861 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:15.415975 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:15.416024 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:15.416192 (+ 168us) server_negotiation.cc:299] Negotiation successful
0925 07:25:15.416244 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237}
I20260925 07:25:15.416440 7214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:15.411000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:39083 (local address 127.5.158.66:33231)
0925 07:25:15.411107 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:15.411116 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:15.411156 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:15.411437 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:15.411438 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:15.411599 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:15.411642 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:15.411643 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.412249 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.412250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:15.412693 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:15.412694 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.413009 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.413010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:15.413071 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:15.413495 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:15.413499 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:15.413794 (+ 295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:15.415277 (+ 1483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:15.415278 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:15.415279 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:15.415540 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:15.415724 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:15.415725 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:15.415726 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:15.415797 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:15.416041 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:15.416042 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:15.416093 (+ 51us) client_negotiation.cc:769] Sending connection context
0925 07:25:15.416179 (+ 86us) client_negotiation.cc:240] Negotiation successful
0925 07:25:15.416300 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":59,"threads_started":1}
WARNING: no policy specified for kudu/127.5.158.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab.
Entry for principal kudu/127.5.158.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab.
I20260925 07:25:15.433921 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.67:0
--local_ip_for_outbound_sockets=127.5.158.67
--webserver_interface=127.5.158.67
--webserver_port=0
--tserver_master_addrs=127.5.158.126:39083
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab
--principal=kudu/127.5.158.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:43541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:25:15.508036 7217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:25:15.508203 7217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:25:15.508226 7217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:25:15.509518 7217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:25:15.509557 7217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:25:15.509577 7217 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 07:25:15.509594 7217 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.67
I20260925 07:25:15.511052 7217 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:43541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab
--principal=kudu/127.5.158.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.158.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.158.126:39083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7217
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.67
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:25:15.511257 7217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:25:15.511452 7217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:25:15.516034 7225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:15.516067 7222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:15.516255 7217 server_base.cc:1034] running on GCE node
W20260925 07:25:15.516352 7223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:15.516543 7217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:25:15.516733 7217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:25:15.517859 7217 hybrid_clock.cc:648] HybridClock initialized: now 1790321115517851 us; error 26 us; skew 500 ppm
Sep 25 07:25:15 dist-test-slave-1drq krb5kdc[6820](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321115, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:25:15.520085 7217 init.cc:377] Logged in from keytab as kudu/127.5.158.67@KRBTEST.COM (short username kudu)
I20260925 07:25:15.520473 7217 webserver.cc:460] Webserver started at http://127.5.158.67:36013/ using document root <none> and password file <none>
I20260925 07:25:15.520684 7217 fs_manager.cc:359] Metadata directory not provided
I20260925 07:25:15.520727 7217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:25:15.520856 7217 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:25:15.521840 7217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/ts-2/data/instance:
uuid: "3617119bd01e4bf98a7ff766231bf137"
format_stamp: "Formatted at 2026-09-25 07:25:15 on dist-test-slave-1drq"
server_key: "6a92daaab12fd2df4ce94bec0ba1a24d"
server_key_iv: "eb7c595a9fb55173fbf44d55fe800b1b"
server_key_version: "encryptionkey@0"
I20260925 07:25:15.522158 7217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/ts-2/wal/instance:
uuid: "3617119bd01e4bf98a7ff766231bf137"
format_stamp: "Formatted at 2026-09-25 07:25:15 on dist-test-slave-1drq"
server_key: "6a92daaab12fd2df4ce94bec0ba1a24d"
server_key_iv: "eb7c595a9fb55173fbf44d55fe800b1b"
server_key_version: "encryptionkey@0"
I20260925 07:25:15.523415 7217 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 07:25:15.524138 7232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:15.524307 7217 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 07:25:15.524396 7217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/ts-2/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/ts-2/wal
uuid: "3617119bd01e4bf98a7ff766231bf137"
format_stamp: "Formatted at 2026-09-25 07:25:15 on dist-test-slave-1drq"
server_key: "6a92daaab12fd2df4ce94bec0ba1a24d"
server_key_iv: "eb7c595a9fb55173fbf44d55fe800b1b"
server_key_version: "encryptionkey@0"
I20260925 07:25:15.524477 7217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:25:15.532254 7217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:25:15.534008 7217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:25:15.534149 7217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:25:15.534574 7217 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:25:15.534629 7217 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:15.534675 7217 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:25:15.534706 7217 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:15.541778 7217 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.67:37691
I20260925 07:25:15.541891 7347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.67:37691 every 8 connection(s)
Sep 25 07:25:15 dist-test-slave-1drq krb5kdc[6820](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321115, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.67@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:25:15.542124 7217 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
I20260925 07:25:15.545424 6924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:15.535114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:58853 (local address 127.5.158.126:39083)
0925 07:25:15.535182 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:15.535184 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:15.535426 (+ 242us) server_negotiation.cc:407] Connection header received
0925 07:25:15.536086 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:15.536087 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:15.536100 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:15.536171 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:15.537053 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.537888 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.539542 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.539867 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.539941 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:15.542635 (+ 2694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:15.542637 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:15.542637 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:15.542644 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:15.544067 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:15.544455 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:15.544457 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:15.544457 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:15.544512 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:15.544758 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:15.544759 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:15.544759 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:15.544874 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:15.544947 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:15.545263 (+ 316us) server_negotiation.cc:299] Negotiation successful
0925 07:25:15.545337 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 07:25:15.545639 7241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:15.535082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:39083 (local address 127.5.158.67:58853)
0925 07:25:15.535285 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:15.535329 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:15.535974 (+ 645us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:15.536246 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:15.536251 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:15.536624 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:15.536923 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:15.536930 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.538000 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.538001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:15.539441 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:15.539445 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.539959 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.539960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:15.540121 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:15.540740 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:15.540750 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:15.542557 (+ 1807us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:15.544166 (+ 1609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:15.544173 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:15.544174 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:15.544380 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:15.544586 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:15.544587 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:15.544588 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:15.544693 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:15.544939 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:15.544945 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:15.545074 (+ 129us) client_negotiation.cc:769] Sending connection context
0925 07:25:15.545238 (+ 164us) client_negotiation.cc:240] Negotiation successful
0925 07:25:15.545379 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":94,"threads_started":1}
I20260925 07:25:15.546054 7348 heartbeater.cc:347] Connected to a master server at 127.5.158.126:39083
I20260925 07:25:15.546132 7348 heartbeater.cc:464] Registering TS with master...
I20260925 07:25:15.546301 7348 heartbeater.cc:511] Master 127.5.158.126:39083 requested a full tablet report, sending...
I20260925 07:25:15.546635 6865 ts_manager.cc:194] Registered new tserver with Master: 3617119bd01e4bf98a7ff766231bf137 (127.5.158.67:37691)
I20260925 07:25:15.547168 6865 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.67@KRBTEST.COM'} at 127.5.158.67:58853
I20260925 07:25:15.547598 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 7217
I20260925 07:25:15.547695 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/ts-2/wal/instance
I20260925 07:25:15.547860 5753 external_mini_cluster.cc:1428] Setting key 40b8f0809b05f8f566c361c6218b8867
I20260925 07:25:15.548704 5753 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 07:25:15.553146 6924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:15.547080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:36327 (local address 127.5.158.126:39083)
0925 07:25:15.547310 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:15.547312 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:15.547326 (+ 14us) server_negotiation.cc:407] Connection header received
0925 07:25:15.547341 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:15.547342 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:15.547348 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:15.547393 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:15.547762 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.548427 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.549290 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.549491 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.549572 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:15.550600 (+ 1028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:15.550601 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:15.550601 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:15.550607 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:15.552001 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:15.552402 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:15.552402 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:15.552403 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:15.552448 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:15.552639 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:15.552640 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:15.552640 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:15.552723 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:15.552769 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:15.552958 (+ 189us) server_negotiation.cc:299] Negotiation successful
0925 07:25:15.553006 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213}
I20260925 07:25:15.553210 7354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:15.547013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:39083 (local address 127.5.158.67:36327)
0925 07:25:15.547126 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:15.547136 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:15.547184 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:15.547400 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:15.547401 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:15.547579 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:15.547632 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:15.547634 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.548534 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.548534 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:15.549190 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:15.549192 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.549575 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.549575 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:15.549646 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:15.550060 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:15.550065 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:15.550480 (+ 415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:15.552081 (+ 1601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:15.552082 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:15.552082 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:15.552321 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:15.552510 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:15.552511 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:15.552511 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:15.552576 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:15.552817 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:15.552818 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:15.552867 (+ 49us) client_negotiation.cc:769] Sending connection context
0925 07:25:15.552943 (+ 76us) client_negotiation.cc:240] Negotiation successful
0925 07:25:15.553035 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":69,"threads_started":1}
Sep 25 07:25:15 dist-test-slave-1drq krb5kdc[6820](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321115, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 07:25:15 dist-test-slave-1drq krb5kdc[6820](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321115, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:25:15.570333 6924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:15.564286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39356 (local address 127.5.158.126:39083)
0925 07:25:15.564335 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:15.564337 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:15.564344 (+ 7us) server_negotiation.cc:407] Connection header received
0925 07:25:15.564402 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:15.564402 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:15.564409 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:15.564451 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:15.564943 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.565402 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.566074 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.566239 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.566283 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:15.568246 (+ 1963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:15.568248 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:15.568248 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:15.568253 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:15.569571 (+ 1318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:15.569848 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:15.569849 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:15.569849 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:15.569891 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:15.570026 (+ 135us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:15.570026 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:15.570027 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:15.570098 (+ 71us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:15.570159 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:15.570238 (+ 79us) server_negotiation.cc:299] Negotiation successful
0925 07:25:15.570284 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260925 07:25:15.571864 6865 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39356:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 07:25:15.573339 6865 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 07:25:15.579222 7362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:15.575431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:36607 (local address 127.0.0.1:36686)
0925 07:25:15.575748 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:15.575759 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:15.575802 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:15.576350 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:15.576351 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:15.576353 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:25:15.576389 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:15.576390 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.577548 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.577549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:15.578310 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:15.578312 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.578937 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.578938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:15.579008 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:15.579019 (+ 11us) client_negotiation.cc:769] Sending connection context
0925 07:25:15.579107 (+ 88us) client_negotiation.cc:240] Negotiation successful
0925 07:25:15.579154 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"mutex_wait_us":5,"thread_start_us":157,"threads_started":1}
I20260925 07:25:15.579239 7367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:15.575754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36686 (local address 127.5.158.65:36607)
0925 07:25:15.576140 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:15.576216 (+ 76us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:15.576226 (+ 10us) server_negotiation.cc:407] Connection header received
0925 07:25:15.576261 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:15.576264 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:15.576278 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:15.576353 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:25:15.576461 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.577467 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.578395 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.578845 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.578934 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:15.578986 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:15.579122 (+ 136us) server_negotiation.cc:299] Negotiation successful
0925 07:25:15.579157 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":76,"threads_started":1}
I20260925 07:25:15.579427 7364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:15.575307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37300 (local address 127.5.158.67:37691)
0925 07:25:15.575617 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:15.575621 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:15.575890 (+ 269us) server_negotiation.cc:407] Connection header received
0925 07:25:15.575927 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:15.575929 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:15.575945 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:15.576036 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:25:15.576156 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.577252 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.578398 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.579040 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.579091 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:15.579142 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:15.579314 (+ 172us) server_negotiation.cc:299] Negotiation successful
0925 07:25:15.579346 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":90,"threads_started":1}
I20260925 07:25:15.579346 7363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:15.575224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.66:40785 (local address 127.0.0.1:55796)
0925 07:25:15.575357 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:15.575393 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:15.575481 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:15.575838 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:15.575841 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:15.575843 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:25:15.575900 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:15.575902 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.576932 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.576933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:15.578314 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:15.578316 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.579141 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.579142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:15.579201 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:15.579204 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 07:25:15.579230 (+ 26us) client_negotiation.cc:240] Negotiation successful
0925 07:25:15.579269 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"mutex_wait_us":4,"thread_start_us":55,"threads_started":1}
I20260925 07:25:15.579622 7365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:15.575402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55796 (local address 127.5.158.66:40785)
0925 07:25:15.575619 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:15.575624 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:15.575635 (+ 11us) server_negotiation.cc:407] Connection header received
0925 07:25:15.575672 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:15.575674 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:15.575689 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:15.575812 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:25:15.575965 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.576853 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.578394 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.579046 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.579408 (+ 362us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:15.579466 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:15.579491 (+ 25us) server_negotiation.cc:299] Negotiation successful
0925 07:25:15.579533 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":110,"threads_started":1}
I20260925 07:25:15.579622 7366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:15.575237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.67:37691 (local address 127.0.0.1:37300)
0925 07:25:15.575616 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:15.575625 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:15.575698 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:15.576031 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:15.576031 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:15.576033 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:25:15.576074 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:15.576075 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.577326 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.577327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:15.578315 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:15.578316 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.579189 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.579189 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:15.579240 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:15.579242 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 07:25:15.579527 (+ 285us) client_negotiation.cc:240] Negotiation successful
0925 07:25:15.579564 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"mutex_wait_us":3,"thread_start_us":65,"threads_started":1}
I20260925 07:25:15.580125 7142 tablet_service.cc:1467] Processing CreateTablet for tablet 7685e9053bae4807a81db85ce6520b72 (DEFAULT_TABLE table=test-table [id=8633b67984d14807a160b31b3d7ec36b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:25:15.580190 7282 tablet_service.cc:1467] Processing CreateTablet for tablet 7685e9053bae4807a81db85ce6520b72 (DEFAULT_TABLE table=test-table [id=8633b67984d14807a160b31b3d7ec36b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:25:15.580173 7002 tablet_service.cc:1467] Processing CreateTablet for tablet 7685e9053bae4807a81db85ce6520b72 (DEFAULT_TABLE table=test-table [id=8633b67984d14807a160b31b3d7ec36b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:25:15.580379 7142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7685e9053bae4807a81db85ce6520b72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:25:15.580506 7282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7685e9053bae4807a81db85ce6520b72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:25:15.580506 7002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7685e9053bae4807a81db85ce6520b72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:25:15.583204 7368 tablet_bootstrap.cc:492] T 7685e9053bae4807a81db85ce6520b72 P 7bb91ac7fc154b169ae6c1b68f9b00d4: Bootstrap starting.
I20260925 07:25:15.584038 7368 tablet_bootstrap.cc:654] T 7685e9053bae4807a81db85ce6520b72 P 7bb91ac7fc154b169ae6c1b68f9b00d4: Neither blocks nor log segments found. Creating new log.
I20260925 07:25:15.584194 7370 tablet_bootstrap.cc:492] T 7685e9053bae4807a81db85ce6520b72 P 74cbfed8f72549e0846b0e9636ad002c: Bootstrap starting.
I20260925 07:25:15.584435 7368 log.cc:824] T 7685e9053bae4807a81db85ce6520b72 P 7bb91ac7fc154b169ae6c1b68f9b00d4: Log is configured to *not* fsync() on all Append() calls
I20260925 07:25:15.584443 7369 tablet_bootstrap.cc:492] T 7685e9053bae4807a81db85ce6520b72 P 3617119bd01e4bf98a7ff766231bf137: Bootstrap starting.
I20260925 07:25:15.584830 7370 tablet_bootstrap.cc:654] T 7685e9053bae4807a81db85ce6520b72 P 74cbfed8f72549e0846b0e9636ad002c: Neither blocks nor log segments found. Creating new log.
I20260925 07:25:15.585224 7370 log.cc:824] T 7685e9053bae4807a81db85ce6520b72 P 74cbfed8f72549e0846b0e9636ad002c: Log is configured to *not* fsync() on all Append() calls
I20260925 07:25:15.585382 7368 tablet_bootstrap.cc:492] T 7685e9053bae4807a81db85ce6520b72 P 7bb91ac7fc154b169ae6c1b68f9b00d4: No bootstrap required, opened a new log
I20260925 07:25:15.585410 7369 tablet_bootstrap.cc:654] T 7685e9053bae4807a81db85ce6520b72 P 3617119bd01e4bf98a7ff766231bf137: Neither blocks nor log segments found. Creating new log.
I20260925 07:25:15.585467 7368 ts_tablet_manager.cc:1397] T 7685e9053bae4807a81db85ce6520b72 P 7bb91ac7fc154b169ae6c1b68f9b00d4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 07:25:15.585774 7369 log.cc:824] T 7685e9053bae4807a81db85ce6520b72 P 3617119bd01e4bf98a7ff766231bf137: Log is configured to *not* fsync() on all Append() calls
I20260925 07:25:15.586095 7370 tablet_bootstrap.cc:492] T 7685e9053bae4807a81db85ce6520b72 P 74cbfed8f72549e0846b0e9636ad002c: No bootstrap required, opened a new log
I20260925 07:25:15.586174 7370 ts_tablet_manager.cc:1397] T 7685e9053bae4807a81db85ce6520b72 P 74cbfed8f72549e0846b0e9636ad002c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 07:25:15.586655 7369 tablet_bootstrap.cc:492] T 7685e9053bae4807a81db85ce6520b72 P 3617119bd01e4bf98a7ff766231bf137: No bootstrap required, opened a new log
I20260925 07:25:15.586745 7369 ts_tablet_manager.cc:1397] T 7685e9053bae4807a81db85ce6520b72 P 3617119bd01e4bf98a7ff766231bf137: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260925 07:25:15.587776 7368 raft_consensus.cc:348] T 7685e9053bae4807a81db85ce6520b72 P 7bb91ac7fc154b169ae6c1b68f9b00d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bb91ac7fc154b169ae6c1b68f9b00d4" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 40785 } } peers { permanent_uuid: "3617119bd01e4bf98a7ff766231bf137" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 37691 } } peers { permanent_uuid: "74cbfed8f72549e0846b0e9636ad002c" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 36607 } }
I20260925 07:25:15.588022 7368 raft_consensus.cc:374] T 7685e9053bae4807a81db85ce6520b72 P 7bb91ac7fc154b169ae6c1b68f9b00d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:25:15.588079 7368 raft_consensus.cc:729] T 7685e9053bae4807a81db85ce6520b72 P 7bb91ac7fc154b169ae6c1b68f9b00d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bb91ac7fc154b169ae6c1b68f9b00d4, State: Initialized, Role: FOLLOWER
I20260925 07:25:15.588204 7368 consensus_queue.cc:260] T 7685e9053bae4807a81db85ce6520b72 P 7bb91ac7fc154b169ae6c1b68f9b00d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bb91ac7fc154b169ae6c1b68f9b00d4" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 40785 } } peers { permanent_uuid: "3617119bd01e4bf98a7ff766231bf137" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 37691 } } peers { permanent_uuid: "74cbfed8f72549e0846b0e9636ad002c" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 36607 } }
I20260925 07:25:15.588171 7370 raft_consensus.cc:348] T 7685e9053bae4807a81db85ce6520b72 P 74cbfed8f72549e0846b0e9636ad002c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bb91ac7fc154b169ae6c1b68f9b00d4" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 40785 } } peers { permanent_uuid: "3617119bd01e4bf98a7ff766231bf137" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 37691 } } peers { permanent_uuid: "74cbfed8f72549e0846b0e9636ad002c" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 36607 } }
I20260925 07:25:15.588320 7370 raft_consensus.cc:374] T 7685e9053bae4807a81db85ce6520b72 P 74cbfed8f72549e0846b0e9636ad002c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:25:15.588363 7370 raft_consensus.cc:729] T 7685e9053bae4807a81db85ce6520b72 P 74cbfed8f72549e0846b0e9636ad002c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74cbfed8f72549e0846b0e9636ad002c, State: Initialized, Role: FOLLOWER
I20260925 07:25:15.588485 7370 consensus_queue.cc:260] T 7685e9053bae4807a81db85ce6520b72 P 74cbfed8f72549e0846b0e9636ad002c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bb91ac7fc154b169ae6c1b68f9b00d4" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 40785 } } peers { permanent_uuid: "3617119bd01e4bf98a7ff766231bf137" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 37691 } } peers { permanent_uuid: "74cbfed8f72549e0846b0e9636ad002c" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 36607 } }
I20260925 07:25:15.588548 7368 ts_tablet_manager.cc:1428] T 7685e9053bae4807a81db85ce6520b72 P 7bb91ac7fc154b169ae6c1b68f9b00d4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 07:25:15.588676 7208 heartbeater.cc:503] Master 127.5.158.126:39083 was elected leader, sending a full tablet report...
I20260925 07:25:15.588724 7370 ts_tablet_manager.cc:1428] T 7685e9053bae4807a81db85ce6520b72 P 74cbfed8f72549e0846b0e9636ad002c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 07:25:15.588713 7369 raft_consensus.cc:348] T 7685e9053bae4807a81db85ce6520b72 P 3617119bd01e4bf98a7ff766231bf137 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bb91ac7fc154b169ae6c1b68f9b00d4" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 40785 } } peers { permanent_uuid: "3617119bd01e4bf98a7ff766231bf137" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 37691 } } peers { permanent_uuid: "74cbfed8f72549e0846b0e9636ad002c" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 36607 } }
I20260925 07:25:15.588862 7369 raft_consensus.cc:374] T 7685e9053bae4807a81db85ce6520b72 P 3617119bd01e4bf98a7ff766231bf137 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:25:15.588896 7369 raft_consensus.cc:729] T 7685e9053bae4807a81db85ce6520b72 P 3617119bd01e4bf98a7ff766231bf137 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3617119bd01e4bf98a7ff766231bf137, State: Initialized, Role: FOLLOWER
I20260925 07:25:15.588985 7068 heartbeater.cc:503] Master 127.5.158.126:39083 was elected leader, sending a full tablet report...
I20260925 07:25:15.589000 7369 consensus_queue.cc:260] T 7685e9053bae4807a81db85ce6520b72 P 3617119bd01e4bf98a7ff766231bf137 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bb91ac7fc154b169ae6c1b68f9b00d4" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 40785 } } peers { permanent_uuid: "3617119bd01e4bf98a7ff766231bf137" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 37691 } } peers { permanent_uuid: "74cbfed8f72549e0846b0e9636ad002c" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 36607 } }
I20260925 07:25:15.589263 7348 heartbeater.cc:503] Master 127.5.158.126:39083 was elected leader, sending a full tablet report...
I20260925 07:25:15.589330 7369 ts_tablet_manager.cc:1428] T 7685e9053bae4807a81db85ce6520b72 P 3617119bd01e4bf98a7ff766231bf137: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
W20260925 07:25:15.657356 7209 tablet.cc:2369] T 7685e9053bae4807a81db85ce6520b72 P 7bb91ac7fc154b169ae6c1b68f9b00d4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 07:25:15.710485 7375 raft_consensus.cc:482] T 7685e9053bae4807a81db85ce6520b72 P 74cbfed8f72549e0846b0e9636ad002c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 07:25:15.710630 7375 raft_consensus.cc:504] T 7685e9053bae4807a81db85ce6520b72 P 74cbfed8f72549e0846b0e9636ad002c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bb91ac7fc154b169ae6c1b68f9b00d4" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 40785 } } peers { permanent_uuid: "3617119bd01e4bf98a7ff766231bf137" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 37691 } } peers { permanent_uuid: "74cbfed8f72549e0846b0e9636ad002c" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 36607 } }
I20260925 07:25:15.711018 7375 leader_election.cc:288] T 7685e9053bae4807a81db85ce6520b72 P 74cbfed8f72549e0846b0e9636ad002c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7bb91ac7fc154b169ae6c1b68f9b00d4 (127.5.158.66:40785), 3617119bd01e4bf98a7ff766231bf137 (127.5.158.67:37691)
I20260925 07:25:15.714813 7365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:15.711231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:47091 (local address 127.5.158.66:40785)
0925 07:25:15.711293 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:15.711295 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:15.711305 (+ 10us) server_negotiation.cc:407] Connection header received
0925 07:25:15.711388 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:15.711389 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:15.711398 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:15.711465 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:25:15.711658 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.712396 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.713889 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.714430 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.714508 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:15.714516 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:15.714701 (+ 185us) server_negotiation.cc:299] Negotiation successful
0925 07:25:15.714747 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 07:25:15.714813 7364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:15.711298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:36309 (local address 127.5.158.67:37691)
0925 07:25:15.711340 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:15.711342 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:15.711438 (+ 96us) server_negotiation.cc:407] Connection header received
0925 07:25:15.711512 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:15.711513 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:15.711526 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:15.711562 (+ 36us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:25:15.711711 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.712391 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.713888 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.714430 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.714508 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:15.714516 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:15.714701 (+ 185us) server_negotiation.cc:299] Negotiation successful
0925 07:25:15.714744 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 07:25:15.714834 6961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:15.711197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.66:40785 (local address 127.5.158.65:47091)
0925 07:25:15.711251 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:15.711262 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:15.711306 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:15.711482 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:15.711483 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:15.711486 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:25:15.711579 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:15.711581 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.712475 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.712475 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:15.713799 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:15.713801 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.714518 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.714519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:15.714625 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:15.714627 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 07:25:15.714680 (+ 53us) client_negotiation.cc:240] Negotiation successful
0925 07:25:15.714724 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":44}
I20260925 07:25:15.714843 7377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:15.711227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.67:37691 (local address 127.5.158.65:36309)
0925 07:25:15.711369 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:15.711378 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:15.711433 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:15.711585 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:15.711586 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:15.711588 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:25:15.711646 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:15.711647 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.712474 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.712475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:15.713799 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:15.713801 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.714518 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.714519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:15.714619 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:15.714623 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 07:25:15.714680 (+ 57us) client_negotiation.cc:240] Negotiation successful
0925 07:25:15.714724 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":89,"thread_start_us":89,"threads_started":1}
I20260925 07:25:15.715373 7302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7685e9053bae4807a81db85ce6520b72" candidate_uuid: "74cbfed8f72549e0846b0e9636ad002c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3617119bd01e4bf98a7ff766231bf137" is_pre_election: true
I20260925 07:25:15.715373 7162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7685e9053bae4807a81db85ce6520b72" candidate_uuid: "74cbfed8f72549e0846b0e9636ad002c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bb91ac7fc154b169ae6c1b68f9b00d4" is_pre_election: true
I20260925 07:25:15.715519 7302 raft_consensus.cc:2454] T 7685e9053bae4807a81db85ce6520b72 P 3617119bd01e4bf98a7ff766231bf137 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74cbfed8f72549e0846b0e9636ad002c in term 0.
I20260925 07:25:15.715519 7162 raft_consensus.cc:2454] T 7685e9053bae4807a81db85ce6520b72 P 7bb91ac7fc154b169ae6c1b68f9b00d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74cbfed8f72549e0846b0e9636ad002c in term 0.
I20260925 07:25:15.715735 6955 leader_election.cc:302] T 7685e9053bae4807a81db85ce6520b72 P 74cbfed8f72549e0846b0e9636ad002c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74cbfed8f72549e0846b0e9636ad002c, 7bb91ac7fc154b169ae6c1b68f9b00d4; no voters:
I20260925 07:25:15.715871 7375 raft_consensus.cc:2781] T 7685e9053bae4807a81db85ce6520b72 P 74cbfed8f72549e0846b0e9636ad002c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 07:25:15.715927 7375 raft_consensus.cc:482] T 7685e9053bae4807a81db85ce6520b72 P 74cbfed8f72549e0846b0e9636ad002c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 07:25:15.715951 7375 raft_consensus.cc:3037] T 7685e9053bae4807a81db85ce6520b72 P 74cbfed8f72549e0846b0e9636ad002c [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:25:15.716876 7375 raft_consensus.cc:504] T 7685e9053bae4807a81db85ce6520b72 P 74cbfed8f72549e0846b0e9636ad002c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bb91ac7fc154b169ae6c1b68f9b00d4" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 40785 } } peers { permanent_uuid: "3617119bd01e4bf98a7ff766231bf137" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 37691 } } peers { permanent_uuid: "74cbfed8f72549e0846b0e9636ad002c" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 36607 } }
I20260925 07:25:15.717013 7375 leader_election.cc:288] T 7685e9053bae4807a81db85ce6520b72 P 74cbfed8f72549e0846b0e9636ad002c [CANDIDATE]: Term 1 election: Requested vote from peers 7bb91ac7fc154b169ae6c1b68f9b00d4 (127.5.158.66:40785), 3617119bd01e4bf98a7ff766231bf137 (127.5.158.67:37691)
I20260925 07:25:15.717168 7162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7685e9053bae4807a81db85ce6520b72" candidate_uuid: "74cbfed8f72549e0846b0e9636ad002c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bb91ac7fc154b169ae6c1b68f9b00d4"
I20260925 07:25:15.717172 7302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7685e9053bae4807a81db85ce6520b72" candidate_uuid: "74cbfed8f72549e0846b0e9636ad002c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3617119bd01e4bf98a7ff766231bf137"
I20260925 07:25:15.717248 7302 raft_consensus.cc:3037] T 7685e9053bae4807a81db85ce6520b72 P 3617119bd01e4bf98a7ff766231bf137 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:25:15.717253 7162 raft_consensus.cc:3037] T 7685e9053bae4807a81db85ce6520b72 P 7bb91ac7fc154b169ae6c1b68f9b00d4 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:25:15.717947 7302 raft_consensus.cc:2454] T 7685e9053bae4807a81db85ce6520b72 P 3617119bd01e4bf98a7ff766231bf137 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74cbfed8f72549e0846b0e9636ad002c in term 1.
I20260925 07:25:15.717958 7162 raft_consensus.cc:2454] T 7685e9053bae4807a81db85ce6520b72 P 7bb91ac7fc154b169ae6c1b68f9b00d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74cbfed8f72549e0846b0e9636ad002c in term 1.
I20260925 07:25:15.718134 6954 leader_election.cc:302] T 7685e9053bae4807a81db85ce6520b72 P 74cbfed8f72549e0846b0e9636ad002c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3617119bd01e4bf98a7ff766231bf137, 74cbfed8f72549e0846b0e9636ad002c; no voters:
I20260925 07:25:15.718231 7375 raft_consensus.cc:2781] T 7685e9053bae4807a81db85ce6520b72 P 74cbfed8f72549e0846b0e9636ad002c [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:25:15.718357 7375 raft_consensus.cc:686] T 7685e9053bae4807a81db85ce6520b72 P 74cbfed8f72549e0846b0e9636ad002c [term 1 LEADER]: Becoming Leader. State: Replica: 74cbfed8f72549e0846b0e9636ad002c, State: Running, Role: LEADER
I20260925 07:25:15.718449 7375 consensus_queue.cc:237] T 7685e9053bae4807a81db85ce6520b72 P 74cbfed8f72549e0846b0e9636ad002c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bb91ac7fc154b169ae6c1b68f9b00d4" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 40785 } } peers { permanent_uuid: "3617119bd01e4bf98a7ff766231bf137" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 37691 } } peers { permanent_uuid: "74cbfed8f72549e0846b0e9636ad002c" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 36607 } }
I20260925 07:25:15.719203 6865 catalog_manager.cc:5495] T 7685e9053bae4807a81db85ce6520b72 P 74cbfed8f72549e0846b0e9636ad002c reported cstate change: term changed from 0 to 1, leader changed from <none> to 74cbfed8f72549e0846b0e9636ad002c (127.5.158.65). New cstate: current_term: 1 leader_uuid: "74cbfed8f72549e0846b0e9636ad002c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bb91ac7fc154b169ae6c1b68f9b00d4" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 40785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3617119bd01e4bf98a7ff766231bf137" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 37691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74cbfed8f72549e0846b0e9636ad002c" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 36607 } health_report { overall_health: HEALTHY } } }
I20260925 07:25:15.759582 7367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:15.757485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36696 (local address 127.5.158.65:36607)
0925 07:25:15.757540 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:15.757542 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:15.757559 (+ 17us) server_negotiation.cc:407] Connection header received
0925 07:25:15.757588 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:15.757589 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:15.757604 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:15.757661 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 07:25:15.757774 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.758226 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.758922 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.759100 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.759146 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:15.759323 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 07:25:15.759405 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 07:25:15.759459 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:15.759500 (+ 41us) server_negotiation.cc:299] Negotiation successful
0925 07:25:15.759516 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260925 07:25:15.761247 7162 raft_consensus.cc:1261] T 7685e9053bae4807a81db85ce6520b72 P 7bb91ac7fc154b169ae6c1b68f9b00d4 [term 1 FOLLOWER]: Refusing update from remote peer 74cbfed8f72549e0846b0e9636ad002c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:25:15.761247 7302 raft_consensus.cc:1261] T 7685e9053bae4807a81db85ce6520b72 P 3617119bd01e4bf98a7ff766231bf137 [term 1 FOLLOWER]: Refusing update from remote peer 74cbfed8f72549e0846b0e9636ad002c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:25:15.761569 7375 consensus_queue.cc:1035] T 7685e9053bae4807a81db85ce6520b72 P 74cbfed8f72549e0846b0e9636ad002c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3617119bd01e4bf98a7ff766231bf137" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 37691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:25:15.761701 7378 consensus_queue.cc:1035] T 7685e9053bae4807a81db85ce6520b72 P 74cbfed8f72549e0846b0e9636ad002c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bb91ac7fc154b169ae6c1b68f9b00d4" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 40785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:25:15.764223 7382 mvcc.cc:205] Tried to move back new op lower bound from 7333155290155175936 to 7333155289983111168. Current Snapshot: MvccSnapshot[applied={T|T < 7333155290155175936}]
I20260925 07:25:15.764416 7383 mvcc.cc:205] Tried to move back new op lower bound from 7333155290155175936 to 7333155289983111168. Current Snapshot: MvccSnapshot[applied={T|T < 7333155290155175936}]
I20260925 07:25:15.767380 6865 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 07:25:15.767468 6865 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 07:25:15.768177 6865 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e959c62d37d34aecaf9bf9e28cba7df9: Sending DeleteTablet for 3 replicas of tablet 7685e9053bae4807a81db85ce6520b72
I20260925 07:25:15.768482 7142 tablet_service.cc:1514] Processing DeleteTablet for tablet 7685e9053bae4807a81db85ce6520b72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:25:15 UTC) from {username='kudu', principal='kudu/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:55796
I20260925 07:25:15.768568 7282 tablet_service.cc:1514] Processing DeleteTablet for tablet 7685e9053bae4807a81db85ce6520b72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:25:15 UTC) from {username='kudu', principal='kudu/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:37300
I20260925 07:25:15.768790 7392 tablet_replica.cc:330] T 7685e9053bae4807a81db85ce6520b72 P 3617119bd01e4bf98a7ff766231bf137: stopping tablet replica
I20260925 07:25:15.768797 6865 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39356:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 07:25:15.768855 7002 tablet_service.cc:1514] Processing DeleteTablet for tablet 7685e9053bae4807a81db85ce6520b72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:25:15 UTC) from {username='kudu', principal='kudu/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:36686
W20260925 07:25:15.768941 6865 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 07:25:15.769052 7393 tablet_replica.cc:330] T 7685e9053bae4807a81db85ce6520b72 P 74cbfed8f72549e0846b0e9636ad002c: stopping tablet replica
I20260925 07:25:15.769166 7392 raft_consensus.cc:2229] T 7685e9053bae4807a81db85ce6520b72 P 3617119bd01e4bf98a7ff766231bf137 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 07:25:15.769290 7392 raft_consensus.cc:2258] T 7685e9053bae4807a81db85ce6520b72 P 3617119bd01e4bf98a7ff766231bf137 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:25:15.769161 7393 raft_consensus.cc:2229] T 7685e9053bae4807a81db85ce6520b72 P 74cbfed8f72549e0846b0e9636ad002c [term 1 LEADER]: Raft consensus shutting down.
I20260925 07:25:15.769734 7393 raft_consensus.cc:2258] T 7685e9053bae4807a81db85ce6520b72 P 74cbfed8f72549e0846b0e9636ad002c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:25:15.769909 7391 tablet_replica.cc:330] T 7685e9053bae4807a81db85ce6520b72 P 7bb91ac7fc154b169ae6c1b68f9b00d4: stopping tablet replica
I20260925 07:25:15.769999 7391 raft_consensus.cc:2229] T 7685e9053bae4807a81db85ce6520b72 P 7bb91ac7fc154b169ae6c1b68f9b00d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 07:25:15.770052 7392 ts_tablet_manager.cc:1905] T 7685e9053bae4807a81db85ce6520b72 P 3617119bd01e4bf98a7ff766231bf137: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:25:15.770107 7391 raft_consensus.cc:2258] T 7685e9053bae4807a81db85ce6520b72 P 7bb91ac7fc154b169ae6c1b68f9b00d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:25:15.770315 7142 tablet_service.cc:1467] Processing CreateTablet for tablet c7ca164441574b95970fb852d202c938 (DEFAULT_TABLE table=test-table [id=94b470169ac24fb8a4a1f0eb0eb100c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:25:15.770313 7002 tablet_service.cc:1467] Processing CreateTablet for tablet c7ca164441574b95970fb852d202c938 (DEFAULT_TABLE table=test-table [id=94b470169ac24fb8a4a1f0eb0eb100c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:25:15.770423 7142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7ca164441574b95970fb852d202c938. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:25:15.770423 7002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7ca164441574b95970fb852d202c938. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:25:15.771724 7370 tablet_bootstrap.cc:492] T c7ca164441574b95970fb852d202c938 P 74cbfed8f72549e0846b0e9636ad002c: Bootstrap starting.
I20260925 07:25:15.771716 7368 tablet_bootstrap.cc:492] T c7ca164441574b95970fb852d202c938 P 7bb91ac7fc154b169ae6c1b68f9b00d4: Bootstrap starting.
I20260925 07:25:15.771826 7282 tablet_service.cc:1467] Processing CreateTablet for tablet c7ca164441574b95970fb852d202c938 (DEFAULT_TABLE table=test-table [id=94b470169ac24fb8a4a1f0eb0eb100c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:25:15.771898 7392 ts_tablet_manager.cc:1918] T 7685e9053bae4807a81db85ce6520b72 P 3617119bd01e4bf98a7ff766231bf137: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 07:25:15.771929 7282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7ca164441574b95970fb852d202c938. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:25:15.771968 7392 log.cc:1196] T 7685e9053bae4807a81db85ce6520b72 P 3617119bd01e4bf98a7ff766231bf137: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/ts-2/wal/wals/7685e9053bae4807a81db85ce6520b72
I20260925 07:25:15.772122 7370 tablet_bootstrap.cc:654] T c7ca164441574b95970fb852d202c938 P 74cbfed8f72549e0846b0e9636ad002c: Neither blocks nor log segments found. Creating new log.
I20260925 07:25:15.772185 7368 tablet_bootstrap.cc:654] T c7ca164441574b95970fb852d202c938 P 7bb91ac7fc154b169ae6c1b68f9b00d4: Neither blocks nor log segments found. Creating new log.
I20260925 07:25:15.772225 7392 ts_tablet_manager.cc:1939] T 7685e9053bae4807a81db85ce6520b72 P 3617119bd01e4bf98a7ff766231bf137: Deleting consensus metadata
I20260925 07:25:15.773203 7369 tablet_bootstrap.cc:492] T c7ca164441574b95970fb852d202c938 P 3617119bd01e4bf98a7ff766231bf137: Bootstrap starting.
I20260925 07:25:15.773615 7369 tablet_bootstrap.cc:654] T c7ca164441574b95970fb852d202c938 P 3617119bd01e4bf98a7ff766231bf137: Neither blocks nor log segments found. Creating new log.
I20260925 07:25:15.773984 6850 catalog_manager.cc:4841] TS 3617119bd01e4bf98a7ff766231bf137 (127.5.158.67:37691): tablet 7685e9053bae4807a81db85ce6520b72 (table test-table [id=8633b67984d14807a160b31b3d7ec36b]) successfully deleted
I20260925 07:25:15.774260 7369 tablet_bootstrap.cc:492] T c7ca164441574b95970fb852d202c938 P 3617119bd01e4bf98a7ff766231bf137: No bootstrap required, opened a new log
I20260925 07:25:15.774307 7369 ts_tablet_manager.cc:1397] T c7ca164441574b95970fb852d202c938 P 3617119bd01e4bf98a7ff766231bf137: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260925 07:25:15.774472 7369 raft_consensus.cc:348] T c7ca164441574b95970fb852d202c938 P 3617119bd01e4bf98a7ff766231bf137 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74cbfed8f72549e0846b0e9636ad002c" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 36607 } } peers { permanent_uuid: "7bb91ac7fc154b169ae6c1b68f9b00d4" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 40785 } } peers { permanent_uuid: "3617119bd01e4bf98a7ff766231bf137" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 37691 } }
I20260925 07:25:15.774537 7369 raft_consensus.cc:374] T c7ca164441574b95970fb852d202c938 P 3617119bd01e4bf98a7ff766231bf137 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:25:15.774559 7369 raft_consensus.cc:729] T c7ca164441574b95970fb852d202c938 P 3617119bd01e4bf98a7ff766231bf137 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3617119bd01e4bf98a7ff766231bf137, State: Initialized, Role: FOLLOWER
I20260925 07:25:15.774605 7369 consensus_queue.cc:260] T c7ca164441574b95970fb852d202c938 P 3617119bd01e4bf98a7ff766231bf137 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74cbfed8f72549e0846b0e9636ad002c" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 36607 } } peers { permanent_uuid: "7bb91ac7fc154b169ae6c1b68f9b00d4" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 40785 } } peers { permanent_uuid: "3617119bd01e4bf98a7ff766231bf137" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 37691 } }
I20260925 07:25:15.774691 7369 ts_tablet_manager.cc:1428] T c7ca164441574b95970fb852d202c938 P 3617119bd01e4bf98a7ff766231bf137: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260925 07:25:15.775390 7391 ts_tablet_manager.cc:1905] T 7685e9053bae4807a81db85ce6520b72 P 7bb91ac7fc154b169ae6c1b68f9b00d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:25:15.775508 7393 ts_tablet_manager.cc:1905] T 7685e9053bae4807a81db85ce6520b72 P 74cbfed8f72549e0846b0e9636ad002c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:25:15.776778 7391 ts_tablet_manager.cc:1918] T 7685e9053bae4807a81db85ce6520b72 P 7bb91ac7fc154b169ae6c1b68f9b00d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 07:25:15.776836 7391 log.cc:1196] T 7685e9053bae4807a81db85ce6520b72 P 7bb91ac7fc154b169ae6c1b68f9b00d4: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/ts-1/wal/wals/7685e9053bae4807a81db85ce6520b72
I20260925 07:25:15.777201 7393 ts_tablet_manager.cc:1918] T 7685e9053bae4807a81db85ce6520b72 P 74cbfed8f72549e0846b0e9636ad002c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 07:25:15.777269 7393 log.cc:1196] T 7685e9053bae4807a81db85ce6520b72 P 74cbfed8f72549e0846b0e9636ad002c: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/ts-0/wal/wals/7685e9053bae4807a81db85ce6520b72
I20260925 07:25:15.777529 7393 ts_tablet_manager.cc:1939] T 7685e9053bae4807a81db85ce6520b72 P 74cbfed8f72549e0846b0e9636ad002c: Deleting consensus metadata
I20260925 07:25:15.777576 7391 ts_tablet_manager.cc:1939] T 7685e9053bae4807a81db85ce6520b72 P 7bb91ac7fc154b169ae6c1b68f9b00d4: Deleting consensus metadata
I20260925 07:25:15.777631 7370 tablet_bootstrap.cc:492] T c7ca164441574b95970fb852d202c938 P 74cbfed8f72549e0846b0e9636ad002c: No bootstrap required, opened a new log
I20260925 07:25:15.777673 7370 ts_tablet_manager.cc:1397] T c7ca164441574b95970fb852d202c938 P 74cbfed8f72549e0846b0e9636ad002c: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260925 07:25:15.777822 7370 raft_consensus.cc:348] T c7ca164441574b95970fb852d202c938 P 74cbfed8f72549e0846b0e9636ad002c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74cbfed8f72549e0846b0e9636ad002c" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 36607 } } peers { permanent_uuid: "7bb91ac7fc154b169ae6c1b68f9b00d4" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 40785 } } peers { permanent_uuid: "3617119bd01e4bf98a7ff766231bf137" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 37691 } }
I20260925 07:25:15.777889 7370 raft_consensus.cc:374] T c7ca164441574b95970fb852d202c938 P 74cbfed8f72549e0846b0e9636ad002c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:25:15.777913 7370 raft_consensus.cc:729] T c7ca164441574b95970fb852d202c938 P 74cbfed8f72549e0846b0e9636ad002c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74cbfed8f72549e0846b0e9636ad002c, State: Initialized, Role: FOLLOWER
I20260925 07:25:15.777967 7370 consensus_queue.cc:260] T c7ca164441574b95970fb852d202c938 P 74cbfed8f72549e0846b0e9636ad002c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74cbfed8f72549e0846b0e9636ad002c" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 36607 } } peers { permanent_uuid: "7bb91ac7fc154b169ae6c1b68f9b00d4" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 40785 } } peers { permanent_uuid: "3617119bd01e4bf98a7ff766231bf137" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 37691 } }
I20260925 07:25:15.778071 7370 ts_tablet_manager.cc:1428] T c7ca164441574b95970fb852d202c938 P 74cbfed8f72549e0846b0e9636ad002c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:15.778198 7368 tablet_bootstrap.cc:492] T c7ca164441574b95970fb852d202c938 P 7bb91ac7fc154b169ae6c1b68f9b00d4: No bootstrap required, opened a new log
I20260925 07:25:15.778245 7368 ts_tablet_manager.cc:1397] T c7ca164441574b95970fb852d202c938 P 7bb91ac7fc154b169ae6c1b68f9b00d4: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260925 07:25:15.778266 6850 catalog_manager.cc:4841] TS 74cbfed8f72549e0846b0e9636ad002c (127.5.158.65:36607): tablet 7685e9053bae4807a81db85ce6520b72 (table test-table [id=8633b67984d14807a160b31b3d7ec36b]) successfully deleted
I20260925 07:25:15.778381 6851 catalog_manager.cc:4841] TS 7bb91ac7fc154b169ae6c1b68f9b00d4 (127.5.158.66:40785): tablet 7685e9053bae4807a81db85ce6520b72 (table test-table [id=8633b67984d14807a160b31b3d7ec36b]) successfully deleted
I20260925 07:25:15.778381 7368 raft_consensus.cc:348] T c7ca164441574b95970fb852d202c938 P 7bb91ac7fc154b169ae6c1b68f9b00d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74cbfed8f72549e0846b0e9636ad002c" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 36607 } } peers { permanent_uuid: "7bb91ac7fc154b169ae6c1b68f9b00d4" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 40785 } } peers { permanent_uuid: "3617119bd01e4bf98a7ff766231bf137" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 37691 } }
I20260925 07:25:15.778448 7368 raft_consensus.cc:374] T c7ca164441574b95970fb852d202c938 P 7bb91ac7fc154b169ae6c1b68f9b00d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:25:15.778470 7368 raft_consensus.cc:729] T c7ca164441574b95970fb852d202c938 P 7bb91ac7fc154b169ae6c1b68f9b00d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bb91ac7fc154b169ae6c1b68f9b00d4, State: Initialized, Role: FOLLOWER
I20260925 07:25:15.778517 7368 consensus_queue.cc:260] T c7ca164441574b95970fb852d202c938 P 7bb91ac7fc154b169ae6c1b68f9b00d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74cbfed8f72549e0846b0e9636ad002c" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 36607 } } peers { permanent_uuid: "7bb91ac7fc154b169ae6c1b68f9b00d4" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 40785 } } peers { permanent_uuid: "3617119bd01e4bf98a7ff766231bf137" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 37691 } }
I20260925 07:25:15.778613 7368 ts_tablet_manager.cc:1428] T c7ca164441574b95970fb852d202c938 P 7bb91ac7fc154b169ae6c1b68f9b00d4: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
W20260925 07:25:15.792687 7349 tablet.cc:2369] T c7ca164441574b95970fb852d202c938 P 3617119bd01e4bf98a7ff766231bf137: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 07:25:15.842499 7386 raft_consensus.cc:482] T c7ca164441574b95970fb852d202c938 P 74cbfed8f72549e0846b0e9636ad002c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 07:25:15.842610 7386 raft_consensus.cc:504] T c7ca164441574b95970fb852d202c938 P 74cbfed8f72549e0846b0e9636ad002c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74cbfed8f72549e0846b0e9636ad002c" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 36607 } } peers { permanent_uuid: "7bb91ac7fc154b169ae6c1b68f9b00d4" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 40785 } } peers { permanent_uuid: "3617119bd01e4bf98a7ff766231bf137" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 37691 } }
I20260925 07:25:15.842821 7386 leader_election.cc:288] T c7ca164441574b95970fb852d202c938 P 74cbfed8f72549e0846b0e9636ad002c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7bb91ac7fc154b169ae6c1b68f9b00d4 (127.5.158.66:40785), 3617119bd01e4bf98a7ff766231bf137 (127.5.158.67:37691)
I20260925 07:25:15.842989 7162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7ca164441574b95970fb852d202c938" candidate_uuid: "74cbfed8f72549e0846b0e9636ad002c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bb91ac7fc154b169ae6c1b68f9b00d4" is_pre_election: true
I20260925 07:25:15.843091 7162 raft_consensus.cc:2454] T c7ca164441574b95970fb852d202c938 P 7bb91ac7fc154b169ae6c1b68f9b00d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74cbfed8f72549e0846b0e9636ad002c in term 0.
I20260925 07:25:15.843019 7302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7ca164441574b95970fb852d202c938" candidate_uuid: "74cbfed8f72549e0846b0e9636ad002c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3617119bd01e4bf98a7ff766231bf137" is_pre_election: true
I20260925 07:25:15.843144 7302 raft_consensus.cc:2454] T c7ca164441574b95970fb852d202c938 P 3617119bd01e4bf98a7ff766231bf137 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74cbfed8f72549e0846b0e9636ad002c in term 0.
I20260925 07:25:15.843277 6955 leader_election.cc:302] T c7ca164441574b95970fb852d202c938 P 74cbfed8f72549e0846b0e9636ad002c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74cbfed8f72549e0846b0e9636ad002c, 7bb91ac7fc154b169ae6c1b68f9b00d4; no voters:
I20260925 07:25:15.843396 7386 raft_consensus.cc:2781] T c7ca164441574b95970fb852d202c938 P 74cbfed8f72549e0846b0e9636ad002c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 07:25:15.843441 7386 raft_consensus.cc:482] T c7ca164441574b95970fb852d202c938 P 74cbfed8f72549e0846b0e9636ad002c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 07:25:15.843464 7386 raft_consensus.cc:3037] T c7ca164441574b95970fb852d202c938 P 74cbfed8f72549e0846b0e9636ad002c [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:25:15.844117 7386 raft_consensus.cc:504] T c7ca164441574b95970fb852d202c938 P 74cbfed8f72549e0846b0e9636ad002c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74cbfed8f72549e0846b0e9636ad002c" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 36607 } } peers { permanent_uuid: "7bb91ac7fc154b169ae6c1b68f9b00d4" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 40785 } } peers { permanent_uuid: "3617119bd01e4bf98a7ff766231bf137" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 37691 } }
I20260925 07:25:15.844251 7386 leader_election.cc:288] T c7ca164441574b95970fb852d202c938 P 74cbfed8f72549e0846b0e9636ad002c [CANDIDATE]: Term 1 election: Requested vote from peers 7bb91ac7fc154b169ae6c1b68f9b00d4 (127.5.158.66:40785), 3617119bd01e4bf98a7ff766231bf137 (127.5.158.67:37691)
I20260925 07:25:15.844386 7162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7ca164441574b95970fb852d202c938" candidate_uuid: "74cbfed8f72549e0846b0e9636ad002c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bb91ac7fc154b169ae6c1b68f9b00d4"
I20260925 07:25:15.844401 7302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7ca164441574b95970fb852d202c938" candidate_uuid: "74cbfed8f72549e0846b0e9636ad002c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3617119bd01e4bf98a7ff766231bf137"
I20260925 07:25:15.844465 7162 raft_consensus.cc:3037] T c7ca164441574b95970fb852d202c938 P 7bb91ac7fc154b169ae6c1b68f9b00d4 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:25:15.844485 7302 raft_consensus.cc:3037] T c7ca164441574b95970fb852d202c938 P 3617119bd01e4bf98a7ff766231bf137 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:25:15.845225 7302 raft_consensus.cc:2454] T c7ca164441574b95970fb852d202c938 P 3617119bd01e4bf98a7ff766231bf137 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74cbfed8f72549e0846b0e9636ad002c in term 1.
I20260925 07:25:15.845283 7162 raft_consensus.cc:2454] T c7ca164441574b95970fb852d202c938 P 7bb91ac7fc154b169ae6c1b68f9b00d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74cbfed8f72549e0846b0e9636ad002c in term 1.
I20260925 07:25:15.845451 6954 leader_election.cc:302] T c7ca164441574b95970fb852d202c938 P 74cbfed8f72549e0846b0e9636ad002c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3617119bd01e4bf98a7ff766231bf137, 74cbfed8f72549e0846b0e9636ad002c; no voters:
I20260925 07:25:15.845556 7386 raft_consensus.cc:2781] T c7ca164441574b95970fb852d202c938 P 74cbfed8f72549e0846b0e9636ad002c [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:25:15.845605 7386 raft_consensus.cc:686] T c7ca164441574b95970fb852d202c938 P 74cbfed8f72549e0846b0e9636ad002c [term 1 LEADER]: Becoming Leader. State: Replica: 74cbfed8f72549e0846b0e9636ad002c, State: Running, Role: LEADER
I20260925 07:25:15.845646 7386 consensus_queue.cc:237] T c7ca164441574b95970fb852d202c938 P 74cbfed8f72549e0846b0e9636ad002c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74cbfed8f72549e0846b0e9636ad002c" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 36607 } } peers { permanent_uuid: "7bb91ac7fc154b169ae6c1b68f9b00d4" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 40785 } } peers { permanent_uuid: "3617119bd01e4bf98a7ff766231bf137" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 37691 } }
I20260925 07:25:15.846220 6865 catalog_manager.cc:5495] T c7ca164441574b95970fb852d202c938 P 74cbfed8f72549e0846b0e9636ad002c reported cstate change: term changed from 0 to 1, leader changed from <none> to 74cbfed8f72549e0846b0e9636ad002c (127.5.158.65). New cstate: current_term: 1 leader_uuid: "74cbfed8f72549e0846b0e9636ad002c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74cbfed8f72549e0846b0e9636ad002c" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 36607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bb91ac7fc154b169ae6c1b68f9b00d4" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 40785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3617119bd01e4bf98a7ff766231bf137" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 37691 } health_report { overall_health: UNKNOWN } } }
Sep 25 07:25:15 dist-test-slave-1drq krb5kdc[6820](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321115, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.126@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:25:15.872383 6924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:15.865795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39370 (local address 127.5.158.126:39083)
0925 07:25:15.865865 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:15.865867 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:15.866001 (+ 134us) server_negotiation.cc:407] Connection header received
0925 07:25:15.866097 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:15.866098 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:15.866111 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:15.866191 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:15.866643 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.867231 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.867886 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.868085 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.868159 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:15.870269 (+ 2110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:15.870270 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:15.870271 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:15.870277 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:15.871405 (+ 1128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:15.871723 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:15.871724 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:15.871725 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:15.871772 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:15.871937 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:15.871938 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:15.871938 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:15.872030 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:15.872098 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:15.872221 (+ 123us) server_negotiation.cc:299] Negotiation successful
0925 07:25:15.872291 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 07:25:15.872469 7403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:15.865733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:39083 (local address 127.0.0.1:39370)
0925 07:25:15.865929 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:15.865939 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:15.866019 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:15.866197 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:15.866198 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:15.866495 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:15.866540 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:15.866542 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.867304 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.867305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:15.867792 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:15.867793 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.868160 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.868161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:15.868225 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:15.868766 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:15.868770 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:15.870191 (+ 1421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:15.871485 (+ 1294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:15.871491 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:15.871493 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:15.871658 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:15.871832 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:15.871833 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:15.871833 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:15.871883 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:15.872084 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:15.872089 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:15.872128 (+ 39us) client_negotiation.cc:769] Sending connection context
0925 07:25:15.872202 (+ 74us) client_negotiation.cc:240] Negotiation successful
0925 07:25:15.872339 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":51,"thread_start_us":99,"threads_started":1}
I20260925 07:25:15.874449 6865 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:39370:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260925 07:25:15.874639 6865 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 07:25:15.875964 7142 tablet_service.cc:1467] Processing CreateTablet for tablet defe9703c51947e9aa9cf27701adbdd6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9789ad3c7cb641c8b65a889523074244]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 07:25:15.876108 7142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet defe9703c51947e9aa9cf27701adbdd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:25:15.877425 7368 tablet_bootstrap.cc:492] T defe9703c51947e9aa9cf27701adbdd6 P 7bb91ac7fc154b169ae6c1b68f9b00d4: Bootstrap starting.
I20260925 07:25:15.877480 7002 tablet_service.cc:1467] Processing CreateTablet for tablet defe9703c51947e9aa9cf27701adbdd6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9789ad3c7cb641c8b65a889523074244]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 07:25:15.877593 7002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet defe9703c51947e9aa9cf27701adbdd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:25:15.877826 7368 tablet_bootstrap.cc:654] T defe9703c51947e9aa9cf27701adbdd6 P 7bb91ac7fc154b169ae6c1b68f9b00d4: Neither blocks nor log segments found. Creating new log.
I20260925 07:25:15.878911 7370 tablet_bootstrap.cc:492] T defe9703c51947e9aa9cf27701adbdd6 P 74cbfed8f72549e0846b0e9636ad002c: Bootstrap starting.
I20260925 07:25:15.878998 7282 tablet_service.cc:1467] Processing CreateTablet for tablet defe9703c51947e9aa9cf27701adbdd6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9789ad3c7cb641c8b65a889523074244]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 07:25:15.879128 7282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet defe9703c51947e9aa9cf27701adbdd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:25:15.879242 7370 tablet_bootstrap.cc:654] T defe9703c51947e9aa9cf27701adbdd6 P 74cbfed8f72549e0846b0e9636ad002c: Neither blocks nor log segments found. Creating new log.
I20260925 07:25:15.879719 7370 tablet_bootstrap.cc:492] T defe9703c51947e9aa9cf27701adbdd6 P 74cbfed8f72549e0846b0e9636ad002c: No bootstrap required, opened a new log
I20260925 07:25:15.879776 7370 ts_tablet_manager.cc:1397] T defe9703c51947e9aa9cf27701adbdd6 P 74cbfed8f72549e0846b0e9636ad002c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 07:25:15.879894 7370 raft_consensus.cc:348] T defe9703c51947e9aa9cf27701adbdd6 P 74cbfed8f72549e0846b0e9636ad002c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74cbfed8f72549e0846b0e9636ad002c" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 36607 } } peers { permanent_uuid: "7bb91ac7fc154b169ae6c1b68f9b00d4" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 40785 } } peers { permanent_uuid: "3617119bd01e4bf98a7ff766231bf137" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 37691 } }
I20260925 07:25:15.879963 7370 raft_consensus.cc:374] T defe9703c51947e9aa9cf27701adbdd6 P 74cbfed8f72549e0846b0e9636ad002c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:25:15.879988 7370 raft_consensus.cc:729] T defe9703c51947e9aa9cf27701adbdd6 P 74cbfed8f72549e0846b0e9636ad002c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74cbfed8f72549e0846b0e9636ad002c, State: Initialized, Role: FOLLOWER
I20260925 07:25:15.880025 7370 consensus_queue.cc:260] T defe9703c51947e9aa9cf27701adbdd6 P 74cbfed8f72549e0846b0e9636ad002c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74cbfed8f72549e0846b0e9636ad002c" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 36607 } } peers { permanent_uuid: "7bb91ac7fc154b169ae6c1b68f9b00d4" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 40785 } } peers { permanent_uuid: "3617119bd01e4bf98a7ff766231bf137" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 37691 } }
I20260925 07:25:15.880139 7370 ts_tablet_manager.cc:1428] T defe9703c51947e9aa9cf27701adbdd6 P 74cbfed8f72549e0846b0e9636ad002c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260925 07:25:15.880219 7386 tablet_replica.cc:439] T defe9703c51947e9aa9cf27701adbdd6 P 74cbfed8f72549e0846b0e9636ad002c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74cbfed8f72549e0846b0e9636ad002c" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 36607 } } peers { permanent_uuid: "7bb91ac7fc154b169ae6c1b68f9b00d4" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 40785 } } peers { permanent_uuid: "3617119bd01e4bf98a7ff766231bf137" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 37691 } } }
I20260925 07:25:15.880285 7386 tablet_replica.cc:442] T defe9703c51947e9aa9cf27701adbdd6 P 74cbfed8f72549e0846b0e9636ad002c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 07:25:15.880474 7369 tablet_bootstrap.cc:492] T defe9703c51947e9aa9cf27701adbdd6 P 3617119bd01e4bf98a7ff766231bf137: Bootstrap starting.
I20260925 07:25:15.880856 7369 tablet_bootstrap.cc:654] T defe9703c51947e9aa9cf27701adbdd6 P 3617119bd01e4bf98a7ff766231bf137: Neither blocks nor log segments found. Creating new log.
I20260925 07:25:15.881505 7369 tablet_bootstrap.cc:492] T defe9703c51947e9aa9cf27701adbdd6 P 3617119bd01e4bf98a7ff766231bf137: No bootstrap required, opened a new log
I20260925 07:25:15.881568 7369 ts_tablet_manager.cc:1397] T defe9703c51947e9aa9cf27701adbdd6 P 3617119bd01e4bf98a7ff766231bf137: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260925 07:25:15.881632 7368 tablet_bootstrap.cc:492] T defe9703c51947e9aa9cf27701adbdd6 P 7bb91ac7fc154b169ae6c1b68f9b00d4: No bootstrap required, opened a new log
I20260925 07:25:15.881675 7368 ts_tablet_manager.cc:1397] T defe9703c51947e9aa9cf27701adbdd6 P 7bb91ac7fc154b169ae6c1b68f9b00d4: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260925 07:25:15.881721 7369 raft_consensus.cc:348] T defe9703c51947e9aa9cf27701adbdd6 P 3617119bd01e4bf98a7ff766231bf137 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74cbfed8f72549e0846b0e9636ad002c" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 36607 } } peers { permanent_uuid: "7bb91ac7fc154b169ae6c1b68f9b00d4" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 40785 } } peers { permanent_uuid: "3617119bd01e4bf98a7ff766231bf137" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 37691 } }
I20260925 07:25:15.881784 7369 raft_consensus.cc:374] T defe9703c51947e9aa9cf27701adbdd6 P 3617119bd01e4bf98a7ff766231bf137 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:25:15.881805 7369 raft_consensus.cc:729] T defe9703c51947e9aa9cf27701adbdd6 P 3617119bd01e4bf98a7ff766231bf137 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3617119bd01e4bf98a7ff766231bf137, State: Initialized, Role: FOLLOWER
I20260925 07:25:15.881830 7368 raft_consensus.cc:348] T defe9703c51947e9aa9cf27701adbdd6 P 7bb91ac7fc154b169ae6c1b68f9b00d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74cbfed8f72549e0846b0e9636ad002c" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 36607 } } peers { permanent_uuid: "7bb91ac7fc154b169ae6c1b68f9b00d4" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 40785 } } peers { permanent_uuid: "3617119bd01e4bf98a7ff766231bf137" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 37691 } }
I20260925 07:25:15.881870 7369 consensus_queue.cc:260] T defe9703c51947e9aa9cf27701adbdd6 P 3617119bd01e4bf98a7ff766231bf137 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74cbfed8f72549e0846b0e9636ad002c" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 36607 } } peers { permanent_uuid: "7bb91ac7fc154b169ae6c1b68f9b00d4" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 40785 } } peers { permanent_uuid: "3617119bd01e4bf98a7ff766231bf137" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 37691 } }
I20260925 07:25:15.881904 7368 raft_consensus.cc:374] T defe9703c51947e9aa9cf27701adbdd6 P 7bb91ac7fc154b169ae6c1b68f9b00d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:25:15.881932 7368 raft_consensus.cc:729] T defe9703c51947e9aa9cf27701adbdd6 P 7bb91ac7fc154b169ae6c1b68f9b00d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bb91ac7fc154b169ae6c1b68f9b00d4, State: Initialized, Role: FOLLOWER
I20260925 07:25:15.881954 7369 ts_tablet_manager.cc:1428] T defe9703c51947e9aa9cf27701adbdd6 P 3617119bd01e4bf98a7ff766231bf137: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:15.881976 7368 consensus_queue.cc:260] T defe9703c51947e9aa9cf27701adbdd6 P 7bb91ac7fc154b169ae6c1b68f9b00d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74cbfed8f72549e0846b0e9636ad002c" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 36607 } } peers { permanent_uuid: "7bb91ac7fc154b169ae6c1b68f9b00d4" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 40785 } } peers { permanent_uuid: "3617119bd01e4bf98a7ff766231bf137" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 37691 } }
I20260925 07:25:15.882052 7368 ts_tablet_manager.cc:1428] T defe9703c51947e9aa9cf27701adbdd6 P 7bb91ac7fc154b169ae6c1b68f9b00d4: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260925 07:25:15.882030 7376 tablet_replica.cc:439] T defe9703c51947e9aa9cf27701adbdd6 P 3617119bd01e4bf98a7ff766231bf137: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74cbfed8f72549e0846b0e9636ad002c" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 36607 } } peers { permanent_uuid: "7bb91ac7fc154b169ae6c1b68f9b00d4" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 40785 } } peers { permanent_uuid: "3617119bd01e4bf98a7ff766231bf137" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 37691 } } }
I20260925 07:25:15.882112 7376 tablet_replica.cc:442] T defe9703c51947e9aa9cf27701adbdd6 P 3617119bd01e4bf98a7ff766231bf137: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 07:25:15.882200 7374 tablet_replica.cc:439] T defe9703c51947e9aa9cf27701adbdd6 P 7bb91ac7fc154b169ae6c1b68f9b00d4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74cbfed8f72549e0846b0e9636ad002c" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 36607 } } peers { permanent_uuid: "7bb91ac7fc154b169ae6c1b68f9b00d4" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 40785 } } peers { permanent_uuid: "3617119bd01e4bf98a7ff766231bf137" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 37691 } } }
I20260925 07:25:15.882333 7374 tablet_replica.cc:442] T defe9703c51947e9aa9cf27701adbdd6 P 7bb91ac7fc154b169ae6c1b68f9b00d4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 07:25:15.892416 7376 raft_consensus.cc:482] T defe9703c51947e9aa9cf27701adbdd6 P 3617119bd01e4bf98a7ff766231bf137 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 07:25:15.892472 7376 raft_consensus.cc:504] T defe9703c51947e9aa9cf27701adbdd6 P 3617119bd01e4bf98a7ff766231bf137 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74cbfed8f72549e0846b0e9636ad002c" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 36607 } } peers { permanent_uuid: "7bb91ac7fc154b169ae6c1b68f9b00d4" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 40785 } } peers { permanent_uuid: "3617119bd01e4bf98a7ff766231bf137" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 37691 } }
I20260925 07:25:15.892746 7376 leader_election.cc:288] T defe9703c51947e9aa9cf27701adbdd6 P 3617119bd01e4bf98a7ff766231bf137 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74cbfed8f72549e0846b0e9636ad002c (127.5.158.65:36607), 7bb91ac7fc154b169ae6c1b68f9b00d4 (127.5.158.66:40785)
I20260925 07:25:15.895222 7367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:15.892931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:34767 (local address 127.5.158.65:36607)
0925 07:25:15.892983 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:15.892984 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:15.892995 (+ 11us) server_negotiation.cc:407] Connection header received
0925 07:25:15.893019 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:15.893020 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:15.893029 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:15.893079 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:25:15.893272 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.893677 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.894572 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.894904 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.894981 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:15.894989 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:15.895134 (+ 145us) server_negotiation.cc:299] Negotiation successful
0925 07:25:15.895165 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 07:25:15.895224 7241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:15.892879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:36607 (local address 127.5.158.67:34767)
0925 07:25:15.892914 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:15.892925 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:15.892960 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:15.893129 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:15.893130 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:15.893132 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:25:15.893196 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:15.893198 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.893782 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.893783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:15.894475 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:15.894476 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.894979 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.894979 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:15.895064 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:15.895067 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 07:25:15.895129 (+ 62us) client_negotiation.cc:240] Negotiation successful
0925 07:25:15.895165 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260925 07:25:15.895484 7022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "defe9703c51947e9aa9cf27701adbdd6" candidate_uuid: "3617119bd01e4bf98a7ff766231bf137" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74cbfed8f72549e0846b0e9636ad002c" is_pre_election: true
I20260925 07:25:15.895632 7022 raft_consensus.cc:2454] T defe9703c51947e9aa9cf27701adbdd6 P 74cbfed8f72549e0846b0e9636ad002c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3617119bd01e4bf98a7ff766231bf137 in term 0.
I20260925 07:25:15.895795 7365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:15.892956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:39385 (local address 127.5.158.66:40785)
0925 07:25:15.893002 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:15.893004 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:15.893169 (+ 165us) server_negotiation.cc:407] Connection header received
0925 07:25:15.893256 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:15.893257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:15.893265 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:15.893316 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:25:15.893456 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.893885 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.895006 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.895456 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.895537 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:15.895545 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:15.895696 (+ 151us) server_negotiation.cc:299] Negotiation successful
0925 07:25:15.895733 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260925 07:25:15.895799 7407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:15.892885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.66:40785 (local address 127.5.158.67:39385)
0925 07:25:15.893121 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:15.893129 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:15.893194 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:15.893319 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:15.893320 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:15.893321 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:25:15.893393 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:15.893395 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.893966 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.893967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:15.894915 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:15.894918 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.895533 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.895534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:15.895627 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:15.895631 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 07:25:15.895687 (+ 56us) client_negotiation.cc:240] Negotiation successful
0925 07:25:15.895724 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"mutex_wait_us":32,"thread_start_us":88,"threads_started":1}
I20260925 07:25:15.895869 7234 leader_election.cc:302] T defe9703c51947e9aa9cf27701adbdd6 P 3617119bd01e4bf98a7ff766231bf137 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3617119bd01e4bf98a7ff766231bf137, 74cbfed8f72549e0846b0e9636ad002c; no voters:
I20260925 07:25:15.895994 7376 raft_consensus.cc:2781] T defe9703c51947e9aa9cf27701adbdd6 P 3617119bd01e4bf98a7ff766231bf137 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 07:25:15.896037 7376 raft_consensus.cc:482] T defe9703c51947e9aa9cf27701adbdd6 P 3617119bd01e4bf98a7ff766231bf137 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 07:25:15.896059 7376 raft_consensus.cc:3037] T defe9703c51947e9aa9cf27701adbdd6 P 3617119bd01e4bf98a7ff766231bf137 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:25:15.896082 7162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "defe9703c51947e9aa9cf27701adbdd6" candidate_uuid: "3617119bd01e4bf98a7ff766231bf137" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bb91ac7fc154b169ae6c1b68f9b00d4" is_pre_election: true
I20260925 07:25:15.896176 7162 raft_consensus.cc:2454] T defe9703c51947e9aa9cf27701adbdd6 P 7bb91ac7fc154b169ae6c1b68f9b00d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3617119bd01e4bf98a7ff766231bf137 in term 0.
I20260925 07:25:15.896770 7376 raft_consensus.cc:504] T defe9703c51947e9aa9cf27701adbdd6 P 3617119bd01e4bf98a7ff766231bf137 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74cbfed8f72549e0846b0e9636ad002c" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 36607 } } peers { permanent_uuid: "7bb91ac7fc154b169ae6c1b68f9b00d4" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 40785 } } peers { permanent_uuid: "3617119bd01e4bf98a7ff766231bf137" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 37691 } }
I20260925 07:25:15.896893 7376 leader_election.cc:288] T defe9703c51947e9aa9cf27701adbdd6 P 3617119bd01e4bf98a7ff766231bf137 [CANDIDATE]: Term 1 election: Requested vote from peers 74cbfed8f72549e0846b0e9636ad002c (127.5.158.65:36607), 7bb91ac7fc154b169ae6c1b68f9b00d4 (127.5.158.66:40785)
I20260925 07:25:15.897032 7022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "defe9703c51947e9aa9cf27701adbdd6" candidate_uuid: "3617119bd01e4bf98a7ff766231bf137" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74cbfed8f72549e0846b0e9636ad002c"
I20260925 07:25:15.897042 7162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "defe9703c51947e9aa9cf27701adbdd6" candidate_uuid: "3617119bd01e4bf98a7ff766231bf137" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bb91ac7fc154b169ae6c1b68f9b00d4"
I20260925 07:25:15.897113 7022 raft_consensus.cc:3037] T defe9703c51947e9aa9cf27701adbdd6 P 74cbfed8f72549e0846b0e9636ad002c [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:25:15.897121 7162 raft_consensus.cc:3037] T defe9703c51947e9aa9cf27701adbdd6 P 7bb91ac7fc154b169ae6c1b68f9b00d4 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:25:15.897732 7162 raft_consensus.cc:2454] T defe9703c51947e9aa9cf27701adbdd6 P 7bb91ac7fc154b169ae6c1b68f9b00d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3617119bd01e4bf98a7ff766231bf137 in term 1.
I20260925 07:25:15.897816 7022 raft_consensus.cc:2454] T defe9703c51947e9aa9cf27701adbdd6 P 74cbfed8f72549e0846b0e9636ad002c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3617119bd01e4bf98a7ff766231bf137 in term 1.
I20260925 07:25:15.897882 7235 leader_election.cc:302] T defe9703c51947e9aa9cf27701adbdd6 P 3617119bd01e4bf98a7ff766231bf137 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3617119bd01e4bf98a7ff766231bf137, 7bb91ac7fc154b169ae6c1b68f9b00d4; no voters:
I20260925 07:25:15.897976 7376 raft_consensus.cc:2781] T defe9703c51947e9aa9cf27701adbdd6 P 3617119bd01e4bf98a7ff766231bf137 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:25:15.898144 7376 raft_consensus.cc:686] T defe9703c51947e9aa9cf27701adbdd6 P 3617119bd01e4bf98a7ff766231bf137 [term 1 LEADER]: Becoming Leader. State: Replica: 3617119bd01e4bf98a7ff766231bf137, State: Running, Role: LEADER
I20260925 07:25:15.898216 7376 consensus_queue.cc:237] T defe9703c51947e9aa9cf27701adbdd6 P 3617119bd01e4bf98a7ff766231bf137 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74cbfed8f72549e0846b0e9636ad002c" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 36607 } } peers { permanent_uuid: "7bb91ac7fc154b169ae6c1b68f9b00d4" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 40785 } } peers { permanent_uuid: "3617119bd01e4bf98a7ff766231bf137" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 37691 } }
I20260925 07:25:15.898520 7408 tablet_replica.cc:439] T defe9703c51947e9aa9cf27701adbdd6 P 3617119bd01e4bf98a7ff766231bf137: TxnStatusTablet state changed. Reason: New leader 3617119bd01e4bf98a7ff766231bf137. Latest consensus state: current_term: 1 leader_uuid: "3617119bd01e4bf98a7ff766231bf137" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74cbfed8f72549e0846b0e9636ad002c" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 36607 } } peers { permanent_uuid: "7bb91ac7fc154b169ae6c1b68f9b00d4" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 40785 } } peers { permanent_uuid: "3617119bd01e4bf98a7ff766231bf137" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 37691 } } }
I20260925 07:25:15.898592 7408 tablet_replica.cc:442] T defe9703c51947e9aa9cf27701adbdd6 P 3617119bd01e4bf98a7ff766231bf137: This TxnStatusTablet replica's current role is: LEADER
I20260925 07:25:15.898728 7410 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260925 07:25:15.898819 7410 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260925 07:25:15.898849 6865 catalog_manager.cc:5495] T defe9703c51947e9aa9cf27701adbdd6 P 3617119bd01e4bf98a7ff766231bf137 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3617119bd01e4bf98a7ff766231bf137 (127.5.158.67). New cstate: current_term: 1 leader_uuid: "3617119bd01e4bf98a7ff766231bf137" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74cbfed8f72549e0846b0e9636ad002c" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 36607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bb91ac7fc154b169ae6c1b68f9b00d4" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 40785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3617119bd01e4bf98a7ff766231bf137" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 37691 } health_report { overall_health: HEALTHY } } }
I20260925 07:25:15.908704 7364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:15.906511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37304 (local address 127.5.158.67:37691)
0925 07:25:15.906554 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:15.906555 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:15.906564 (+ 9us) server_negotiation.cc:407] Connection header received
0925 07:25:15.906636 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:15.906637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:15.906651 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:15.906711 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 07:25:15.906866 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.907305 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.908014 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.908190 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.908251 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:15.908429 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 07:25:15.908514 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 07:25:15.908587 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:15.908638 (+ 51us) server_negotiation.cc:299] Negotiation successful
0925 07:25:15.908654 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 07:25:15.908746 7403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:15.906442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.67:37691 (local address 127.0.0.1:37304)
0925 07:25:15.906508 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:15.906518 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:15.906561 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:15.906717 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:15.906717 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:15.906718 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 07:25:15.906799 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:15.906802 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.907385 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.907386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:15.907924 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:15.907926 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.908263 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.908264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:15.908331 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:15.908350 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 07:25:15.908576 (+ 226us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 07:25:15.908584 (+ 8us) client_negotiation.cc:769] Sending connection context
0925 07:25:15.908634 (+ 50us) client_negotiation.cc:240] Negotiation successful
0925 07:25:15.908677 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260925 07:25:15.909638 7162 raft_consensus.cc:1261] T defe9703c51947e9aa9cf27701adbdd6 P 7bb91ac7fc154b169ae6c1b68f9b00d4 [term 1 FOLLOWER]: Refusing update from remote peer 3617119bd01e4bf98a7ff766231bf137: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:25:15.909734 7022 raft_consensus.cc:1261] T defe9703c51947e9aa9cf27701adbdd6 P 74cbfed8f72549e0846b0e9636ad002c [term 1 FOLLOWER]: Refusing update from remote peer 3617119bd01e4bf98a7ff766231bf137: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:25:15.909961 7408 consensus_queue.cc:1035] T defe9703c51947e9aa9cf27701adbdd6 P 3617119bd01e4bf98a7ff766231bf137 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74cbfed8f72549e0846b0e9636ad002c" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 36607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:25:15.910058 7376 consensus_queue.cc:1035] T defe9703c51947e9aa9cf27701adbdd6 P 3617119bd01e4bf98a7ff766231bf137 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bb91ac7fc154b169ae6c1b68f9b00d4" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 40785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:25:15.910449 7374 tablet_replica.cc:439] T defe9703c51947e9aa9cf27701adbdd6 P 7bb91ac7fc154b169ae6c1b68f9b00d4: TxnStatusTablet state changed. Reason: New leader 3617119bd01e4bf98a7ff766231bf137. Latest consensus state: current_term: 1 leader_uuid: "3617119bd01e4bf98a7ff766231bf137" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74cbfed8f72549e0846b0e9636ad002c" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 36607 } } peers { permanent_uuid: "7bb91ac7fc154b169ae6c1b68f9b00d4" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 40785 } } peers { permanent_uuid: "3617119bd01e4bf98a7ff766231bf137" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 37691 } } }
I20260925 07:25:15.910542 7374 tablet_replica.cc:442] T defe9703c51947e9aa9cf27701adbdd6 P 7bb91ac7fc154b169ae6c1b68f9b00d4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 07:25:15.910794 7386 tablet_replica.cc:439] T defe9703c51947e9aa9cf27701adbdd6 P 74cbfed8f72549e0846b0e9636ad002c: TxnStatusTablet state changed. Reason: New leader 3617119bd01e4bf98a7ff766231bf137. Latest consensus state: current_term: 1 leader_uuid: "3617119bd01e4bf98a7ff766231bf137" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74cbfed8f72549e0846b0e9636ad002c" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 36607 } } peers { permanent_uuid: "7bb91ac7fc154b169ae6c1b68f9b00d4" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 40785 } } peers { permanent_uuid: "3617119bd01e4bf98a7ff766231bf137" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 37691 } } }
I20260925 07:25:15.910871 7386 tablet_replica.cc:442] T defe9703c51947e9aa9cf27701adbdd6 P 74cbfed8f72549e0846b0e9636ad002c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 07:25:15.911345 7376 tablet_replica.cc:439] T defe9703c51947e9aa9cf27701adbdd6 P 3617119bd01e4bf98a7ff766231bf137: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3617119bd01e4bf98a7ff766231bf137" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74cbfed8f72549e0846b0e9636ad002c" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 36607 } } peers { permanent_uuid: "7bb91ac7fc154b169ae6c1b68f9b00d4" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 40785 } } peers { permanent_uuid: "3617119bd01e4bf98a7ff766231bf137" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 37691 } } }
I20260925 07:25:15.911408 7376 tablet_replica.cc:442] T defe9703c51947e9aa9cf27701adbdd6 P 3617119bd01e4bf98a7ff766231bf137: This TxnStatusTablet replica's current role is: LEADER
I20260925 07:25:15.911636 7376 tablet_replica.cc:439] T defe9703c51947e9aa9cf27701adbdd6 P 3617119bd01e4bf98a7ff766231bf137: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3617119bd01e4bf98a7ff766231bf137" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74cbfed8f72549e0846b0e9636ad002c" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 36607 } } peers { permanent_uuid: "7bb91ac7fc154b169ae6c1b68f9b00d4" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 40785 } } peers { permanent_uuid: "3617119bd01e4bf98a7ff766231bf137" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 37691 } } }
I20260925 07:25:15.911701 7376 tablet_replica.cc:442] T defe9703c51947e9aa9cf27701adbdd6 P 3617119bd01e4bf98a7ff766231bf137: This TxnStatusTablet replica's current role is: LEADER
I20260925 07:25:15.912014 7386 tablet_replica.cc:439] T defe9703c51947e9aa9cf27701adbdd6 P 74cbfed8f72549e0846b0e9636ad002c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3617119bd01e4bf98a7ff766231bf137" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74cbfed8f72549e0846b0e9636ad002c" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 36607 } } peers { permanent_uuid: "7bb91ac7fc154b169ae6c1b68f9b00d4" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 40785 } } peers { permanent_uuid: "3617119bd01e4bf98a7ff766231bf137" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 37691 } } }
I20260925 07:25:15.912084 7386 tablet_replica.cc:442] T defe9703c51947e9aa9cf27701adbdd6 P 74cbfed8f72549e0846b0e9636ad002c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 07:25:15.912034 7381 mvcc.cc:205] Tried to move back new op lower bound from 7333155290764406784 to 7333155290719408128. Current Snapshot: MvccSnapshot[applied={T|T < 7333155290764406784}]
I20260925 07:25:15.912640 7374 tablet_replica.cc:439] T defe9703c51947e9aa9cf27701adbdd6 P 7bb91ac7fc154b169ae6c1b68f9b00d4: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3617119bd01e4bf98a7ff766231bf137" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74cbfed8f72549e0846b0e9636ad002c" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 36607 } } peers { permanent_uuid: "7bb91ac7fc154b169ae6c1b68f9b00d4" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 40785 } } peers { permanent_uuid: "3617119bd01e4bf98a7ff766231bf137" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 37691 } } }
I20260925 07:25:15.912710 7374 tablet_replica.cc:442] T defe9703c51947e9aa9cf27701adbdd6 P 7bb91ac7fc154b169ae6c1b68f9b00d4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 07:25:15.916937 7364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:15.914581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:37257 (local address 127.5.158.67:37691)
0925 07:25:15.914612 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:15.914613 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:15.914681 (+ 68us) server_negotiation.cc:407] Connection header received
0925 07:25:15.914745 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:15.914745 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:15.914751 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:15.914788 (+ 37us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 07:25:15.914888 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.915242 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.916318 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.916478 (+ 160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.916524 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:15.916731 (+ 207us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 07:25:15.916751 (+ 20us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 07:25:15.916811 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:15.916891 (+ 80us) server_negotiation.cc:299] Negotiation successful
0925 07:25:15.916903 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 07:25:15.917011 7418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:15.914489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.67:37691 (local address 127.5.158.65:37257)
0925 07:25:15.914627 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:15.914637 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:15.914695 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:15.914799 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:15.914800 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:15.914801 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 07:25:15.914838 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:15.914840 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.915312 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.915313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:15.916239 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:15.916241 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.916547 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.916548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:15.916614 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:15.916628 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 07:25:15.916833 (+ 205us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 07:25:15.916834 (+ 1us) client_negotiation.cc:769] Sending connection context
0925 07:25:15.916890 (+ 56us) client_negotiation.cc:240] Negotiation successful
0925 07:25:15.916929 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":83,"threads_started":1}
I20260925 07:25:15.918764 7302 raft_consensus.cc:1261] T c7ca164441574b95970fb852d202c938 P 3617119bd01e4bf98a7ff766231bf137 [term 1 FOLLOWER]: Refusing update from remote peer 74cbfed8f72549e0846b0e9636ad002c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:25:15.918772 7162 raft_consensus.cc:1261] T c7ca164441574b95970fb852d202c938 P 7bb91ac7fc154b169ae6c1b68f9b00d4 [term 1 FOLLOWER]: Refusing update from remote peer 74cbfed8f72549e0846b0e9636ad002c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:25:15.918912 7386 consensus_queue.cc:1035] T c7ca164441574b95970fb852d202c938 P 74cbfed8f72549e0846b0e9636ad002c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bb91ac7fc154b169ae6c1b68f9b00d4" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 40785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:25:15.918979 7378 consensus_queue.cc:1035] T c7ca164441574b95970fb852d202c938 P 74cbfed8f72549e0846b0e9636ad002c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3617119bd01e4bf98a7ff766231bf137" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 37691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:25:15.926000 7354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:15.923380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:36607 (local address 127.5.158.67:36199)
0925 07:25:15.923465 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:15.923473 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:15.923504 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:15.923736 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:15.923736 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:15.923745 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 07:25:15.923787 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:15.923788 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.924547 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.924548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:15.925218 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:15.925220 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.925685 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.925686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:15.925732 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:15.925743 (+ 11us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 07:25:15.925902 (+ 159us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 07:25:15.925903 (+ 1us) client_negotiation.cc:769] Sending connection context
0925 07:25:15.925931 (+ 28us) client_negotiation.cc:240] Negotiation successful
0925 07:25:15.925958 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260925 07:25:15.926013 7367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:15.923607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:36199 (local address 127.5.158.65:36607)
0925 07:25:15.923636 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:15.923637 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:15.923643 (+ 6us) server_negotiation.cc:407] Connection header received
0925 07:25:15.923661 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:15.923661 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:15.923668 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:15.923709 (+ 41us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 07:25:15.923850 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.924481 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.925307 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.925600 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.925680 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:15.925818 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 07:25:15.925848 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 07:25:15.925903 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:15.925934 (+ 31us) server_negotiation.cc:299] Negotiation successful
0925 07:25:15.925951 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260925 07:25:15.934365 6865 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 07:25:15.934448 6865 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 07:25:15.934981 6865 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e959c62d37d34aecaf9bf9e28cba7df9: Sending DeleteTablet for 3 replicas of tablet c7ca164441574b95970fb852d202c938
I20260925 07:25:15.935190 7002 tablet_service.cc:1514] Processing DeleteTablet for tablet c7ca164441574b95970fb852d202c938 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:25:15 UTC) from {username='kudu', principal='kudu/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:36686
I20260925 07:25:15.935222 7142 tablet_service.cc:1514] Processing DeleteTablet for tablet c7ca164441574b95970fb852d202c938 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:25:15 UTC) from {username='kudu', principal='kudu/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:55796
I20260925 07:25:15.935317 7391 tablet_replica.cc:330] T c7ca164441574b95970fb852d202c938 P 7bb91ac7fc154b169ae6c1b68f9b00d4: stopping tablet replica
I20260925 07:25:15.935297 7393 tablet_replica.cc:330] T c7ca164441574b95970fb852d202c938 P 74cbfed8f72549e0846b0e9636ad002c: stopping tablet replica
I20260925 07:25:15.935369 7391 raft_consensus.cc:2229] T c7ca164441574b95970fb852d202c938 P 7bb91ac7fc154b169ae6c1b68f9b00d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 07:25:15.935372 7393 raft_consensus.cc:2229] T c7ca164441574b95970fb852d202c938 P 74cbfed8f72549e0846b0e9636ad002c [term 1 LEADER]: Raft consensus shutting down.
I20260925 07:25:15.935400 7391 raft_consensus.cc:2258] T c7ca164441574b95970fb852d202c938 P 7bb91ac7fc154b169ae6c1b68f9b00d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:25:15.935420 7393 raft_consensus.cc:2258] T c7ca164441574b95970fb852d202c938 P 74cbfed8f72549e0846b0e9636ad002c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:25:15.935592 7282 tablet_service.cc:1514] Processing DeleteTablet for tablet c7ca164441574b95970fb852d202c938 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:25:15 UTC) from {username='kudu', principal='kudu/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:37300
I20260925 07:25:15.935662 7392 tablet_replica.cc:330] T c7ca164441574b95970fb852d202c938 P 3617119bd01e4bf98a7ff766231bf137: stopping tablet replica
I20260925 07:25:15.935668 7391 ts_tablet_manager.cc:1905] T c7ca164441574b95970fb852d202c938 P 7bb91ac7fc154b169ae6c1b68f9b00d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:25:15.935699 7392 raft_consensus.cc:2229] T c7ca164441574b95970fb852d202c938 P 3617119bd01e4bf98a7ff766231bf137 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 07:25:15.935729 7392 raft_consensus.cc:2258] T c7ca164441574b95970fb852d202c938 P 3617119bd01e4bf98a7ff766231bf137 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:25:15.936054 7392 ts_tablet_manager.cc:1905] T c7ca164441574b95970fb852d202c938 P 3617119bd01e4bf98a7ff766231bf137: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:25:15.937175 7391 ts_tablet_manager.cc:1918] T c7ca164441574b95970fb852d202c938 P 7bb91ac7fc154b169ae6c1b68f9b00d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 07:25:15.937222 7391 log.cc:1196] T c7ca164441574b95970fb852d202c938 P 7bb91ac7fc154b169ae6c1b68f9b00d4: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/ts-1/wal/wals/c7ca164441574b95970fb852d202c938
I20260925 07:25:15.937426 7391 ts_tablet_manager.cc:1939] T c7ca164441574b95970fb852d202c938 P 7bb91ac7fc154b169ae6c1b68f9b00d4: Deleting consensus metadata
I20260925 07:25:15.937450 7392 ts_tablet_manager.cc:1918] T c7ca164441574b95970fb852d202c938 P 3617119bd01e4bf98a7ff766231bf137: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 07:25:15.937493 7392 log.cc:1196] T c7ca164441574b95970fb852d202c938 P 3617119bd01e4bf98a7ff766231bf137: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/ts-2/wal/wals/c7ca164441574b95970fb852d202c938
I20260925 07:25:15.937674 7392 ts_tablet_manager.cc:1939] T c7ca164441574b95970fb852d202c938 P 3617119bd01e4bf98a7ff766231bf137: Deleting consensus metadata
I20260925 07:25:15.937775 7393 ts_tablet_manager.cc:1905] T c7ca164441574b95970fb852d202c938 P 74cbfed8f72549e0846b0e9636ad002c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:25:15.937915 6850 catalog_manager.cc:4841] TS 3617119bd01e4bf98a7ff766231bf137 (127.5.158.67:37691): tablet c7ca164441574b95970fb852d202c938 (table test-table [id=94b470169ac24fb8a4a1f0eb0eb100c8]) successfully deleted
I20260925 07:25:15.938045 6851 catalog_manager.cc:4841] TS 7bb91ac7fc154b169ae6c1b68f9b00d4 (127.5.158.66:40785): tablet c7ca164441574b95970fb852d202c938 (table test-table [id=94b470169ac24fb8a4a1f0eb0eb100c8]) successfully deleted
I20260925 07:25:15.939150 7393 ts_tablet_manager.cc:1918] T c7ca164441574b95970fb852d202c938 P 74cbfed8f72549e0846b0e9636ad002c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 07:25:15.939209 7393 log.cc:1196] T c7ca164441574b95970fb852d202c938 P 74cbfed8f72549e0846b0e9636ad002c: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790321110323796-5753-0/minicluster-data/ts-0/wal/wals/c7ca164441574b95970fb852d202c938
I20260925 07:25:15.939409 7393 ts_tablet_manager.cc:1939] T c7ca164441574b95970fb852d202c938 P 74cbfed8f72549e0846b0e9636ad002c: Deleting consensus metadata
I20260925 07:25:15.939760 6850 catalog_manager.cc:4841] TS 74cbfed8f72549e0846b0e9636ad002c (127.5.158.65:36607): tablet c7ca164441574b95970fb852d202c938 (table test-table [id=94b470169ac24fb8a4a1f0eb0eb100c8]) successfully deleted
Sep 25 07:25:15 dist-test-slave-1drq krb5kdc[6820](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321115, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.158.65@KRBTEST.COM
I20260925 07:25:15.946080 7367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:15.938264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36712 (local address 127.5.158.65:36607)
0925 07:25:15.938294 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:15.938295 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:15.938561 (+ 266us) server_negotiation.cc:407] Connection header received
0925 07:25:15.938603 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:15.938604 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:15.938614 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:15.938666 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:15.939325 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.939919 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.940602 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.940880 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.940948 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:15.943410 (+ 2462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:15.943416 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:15.943417 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:15.943423 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:15.944934 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:15.945266 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:15.945269 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:15.945269 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:15.945330 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:15.945530 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:15.945531 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:15.945531 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:15.945692 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:15.945822 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:15.945919 (+ 97us) server_negotiation.cc:299] Negotiation successful
0925 07:25:15.945995 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
W20260925 07:25:15.946393 7062 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36712
I20260925 07:25:15.952955 6924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:15.947285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39380 (local address 127.5.158.126:39083)
0925 07:25:15.947334 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:15.947335 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:15.947405 (+ 70us) server_negotiation.cc:407] Connection header received
0925 07:25:15.947463 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:15.947463 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:15.947472 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:15.947527 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:15.947995 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.948684 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.949419 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:15.949707 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:15.949770 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:15.950750 (+ 980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:15.950751 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:15.950751 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:15.950758 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:15.952068 (+ 1310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:15.952333 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:15.952334 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:15.952335 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:15.952387 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:15.952552 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:15.952553 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:15.952553 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:15.952674 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:15.952755 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:15.952816 (+ 61us) server_negotiation.cc:299] Negotiation successful
0925 07:25:15.952881 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260925 07:25:15.953128 6865 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39380
I20260925 07:25:15.953663 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 6937
I20260925 07:25:15.961849 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 7077
I20260925 07:25:15.966737 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 7217
I20260925 07:25:15.971401 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 6833
2026-09-25T07:25:15Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2061 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 07:25:16 dist-test-slave-1drq krb5kdc[7430](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 07:25:16 dist-test-slave-1drq krb5kdc[7430](info): set up 2 sockets
Sep 25 07:25:16 dist-test-slave-1drq krb5kdc[7430](info): commencing operation
krb5kdc: starting...
W20260925 07:25:16.916651 5753 mini_kdc.cc:121] Time spent starting KDC: real 0.926s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Sep 25 07:25:16 dist-test-slave-1drq krb5kdc[7430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321116, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T07:25:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T07:25:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
I20260925 07:25:17.020246 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:37263
--webserver_interface=127.5.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:34517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.158.126:37263
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:25:17.095177 7443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:25:17.095378 7443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:25:17.095424 7443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:25:17.096521 7443 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:25:17.096581 7443 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260925 07:25:17.096604 7443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:25:17.096627 7443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:25:17.096655 7443 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:25:17.096681 7443 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:25:17.098088 7443 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:34517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.158.126:37263
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:37263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7443
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:25:17.098325 7443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:25:17.098515 7443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:25:17.102890 7448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:17.103125 7443 server_base.cc:1034] running on GCE node
W20260925 07:25:17.102998 7449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:17.102962 7451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:17.103520 7443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:25:17.103806 7443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:25:17.104977 7443 hybrid_clock.cc:648] HybridClock initialized: now 1790321117104969 us; error 38 us; skew 500 ppm
Sep 25 07:25:17 dist-test-slave-1drq krb5kdc[7430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321117, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:25:17.107285 7443 init.cc:377] Logged in from keytab as kudu/127.5.158.126@KRBTEST.COM (short username kudu)
I20260925 07:25:17.107681 7443 webserver.cc:460] Webserver started at http://127.5.158.126:35099/ using document root <none> and password file <none>
I20260925 07:25:17.107847 7443 fs_manager.cc:359] Metadata directory not provided
I20260925 07:25:17.107892 7443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:25:17.107970 7443 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:25:17.108794 7443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/master-0/data/instance:
uuid: "39542b3be6b7448ab00f36a31d0c7915"
format_stamp: "Formatted at 2026-09-25 07:25:17 on dist-test-slave-1drq"
server_key: "41c7d09105e1bfba037a3b5eefb02815"
server_key_iv: "d8bd0cf2c3e2596c51ff4182f6901034"
server_key_version: "encryptionkey@0"
I20260925 07:25:17.109057 7443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/master-0/wal/instance:
uuid: "39542b3be6b7448ab00f36a31d0c7915"
format_stamp: "Formatted at 2026-09-25 07:25:17 on dist-test-slave-1drq"
server_key: "41c7d09105e1bfba037a3b5eefb02815"
server_key_iv: "d8bd0cf2c3e2596c51ff4182f6901034"
server_key_version: "encryptionkey@0"
I20260925 07:25:17.110211 7443 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 07:25:17.110764 7458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:17.110921 7443 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 07:25:17.110993 7443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "39542b3be6b7448ab00f36a31d0c7915"
format_stamp: "Formatted at 2026-09-25 07:25:17 on dist-test-slave-1drq"
server_key: "41c7d09105e1bfba037a3b5eefb02815"
server_key_iv: "d8bd0cf2c3e2596c51ff4182f6901034"
server_key_version: "encryptionkey@0"
I20260925 07:25:17.111042 7443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:25:17.128252 7443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:25:17.130139 7443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:25:17.130231 7443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:25:17.133975 7443 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.126:37263
I20260925 07:25:17.134059 7520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.126:37263 every 8 connection(s)
I20260925 07:25:17.134310 7443 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:25:17.134859 7521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:25:17.136790 7521 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39542b3be6b7448ab00f36a31d0c7915: Bootstrap starting.
I20260925 07:25:17.137337 7521 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 39542b3be6b7448ab00f36a31d0c7915: Neither blocks nor log segments found. Creating new log.
I20260925 07:25:17.137601 7521 log.cc:824] T 00000000000000000000000000000000 P 39542b3be6b7448ab00f36a31d0c7915: Log is configured to *not* fsync() on all Append() calls
I20260925 07:25:17.138188 7521 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39542b3be6b7448ab00f36a31d0c7915: No bootstrap required, opened a new log
I20260925 07:25:17.140018 7521 raft_consensus.cc:348] T 00000000000000000000000000000000 P 39542b3be6b7448ab00f36a31d0c7915 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39542b3be6b7448ab00f36a31d0c7915" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37263 } }
I20260925 07:25:17.140127 7521 raft_consensus.cc:374] T 00000000000000000000000000000000 P 39542b3be6b7448ab00f36a31d0c7915 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:25:17.140147 7521 raft_consensus.cc:729] T 00000000000000000000000000000000 P 39542b3be6b7448ab00f36a31d0c7915 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39542b3be6b7448ab00f36a31d0c7915, State: Initialized, Role: FOLLOWER
I20260925 07:25:17.140295 7521 consensus_queue.cc:260] T 00000000000000000000000000000000 P 39542b3be6b7448ab00f36a31d0c7915 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39542b3be6b7448ab00f36a31d0c7915" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37263 } }
I20260925 07:25:17.140350 7521 raft_consensus.cc:388] T 00000000000000000000000000000000 P 39542b3be6b7448ab00f36a31d0c7915 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 07:25:17.140369 7521 raft_consensus.cc:482] T 00000000000000000000000000000000 P 39542b3be6b7448ab00f36a31d0c7915 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 07:25:17.140442 7521 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 39542b3be6b7448ab00f36a31d0c7915 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:25:17.140877 7521 raft_consensus.cc:504] T 00000000000000000000000000000000 P 39542b3be6b7448ab00f36a31d0c7915 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39542b3be6b7448ab00f36a31d0c7915" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37263 } }
I20260925 07:25:17.140975 7521 leader_election.cc:302] T 00000000000000000000000000000000 P 39542b3be6b7448ab00f36a31d0c7915 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 39542b3be6b7448ab00f36a31d0c7915; no voters:
I20260925 07:25:17.141149 7521 leader_election.cc:288] T 00000000000000000000000000000000 P 39542b3be6b7448ab00f36a31d0c7915 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 07:25:17.141175 7524 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 39542b3be6b7448ab00f36a31d0c7915 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:25:17.141311 7524 raft_consensus.cc:686] T 00000000000000000000000000000000 P 39542b3be6b7448ab00f36a31d0c7915 [term 1 LEADER]: Becoming Leader. State: Replica: 39542b3be6b7448ab00f36a31d0c7915, State: Running, Role: LEADER
I20260925 07:25:17.141369 7521 sys_catalog.cc:564] T 00000000000000000000000000000000 P 39542b3be6b7448ab00f36a31d0c7915 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:25:17.141430 7524 consensus_queue.cc:237] T 00000000000000000000000000000000 P 39542b3be6b7448ab00f36a31d0c7915 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39542b3be6b7448ab00f36a31d0c7915" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37263 } }
I20260925 07:25:17.141762 7525 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39542b3be6b7448ab00f36a31d0c7915 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "39542b3be6b7448ab00f36a31d0c7915" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39542b3be6b7448ab00f36a31d0c7915" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37263 } } }
I20260925 07:25:17.141839 7525 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39542b3be6b7448ab00f36a31d0c7915 [sys.catalog]: This master's current role is: LEADER
I20260925 07:25:17.141994 7526 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39542b3be6b7448ab00f36a31d0c7915 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 39542b3be6b7448ab00f36a31d0c7915. Latest consensus state: current_term: 1 leader_uuid: "39542b3be6b7448ab00f36a31d0c7915" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39542b3be6b7448ab00f36a31d0c7915" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37263 } } }
I20260925 07:25:17.142024 7532 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 07:25:17.142095 7526 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39542b3be6b7448ab00f36a31d0c7915 [sys.catalog]: This master's current role is: LEADER
I20260925 07:25:17.142616 7532 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 07:25:17.143961 7532 catalog_manager.cc:1347] Generated new cluster ID: 2156f50c2bd24218b2b41401b716451f
I20260925 07:25:17.144009 7532 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 07:25:17.144134 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 7443
I20260925 07:25:17.144193 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/master-0/wal/instance
I20260925 07:25:17.144294 5753 external_mini_cluster.cc:1428] Setting key 6bedfabb2fcb959029501174c59a023f
Sep 25 07:25:17 dist-test-slave-1drq krb5kdc[7430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321116, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:25:17.152995 7532 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 07:25:17.153401 7544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:17.144624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47416 (local address 127.5.158.126:37263)
0925 07:25:17.144759 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:17.144766 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:17.145399 (+ 633us) server_negotiation.cc:407] Connection header received
0925 07:25:17.145521 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:17.145526 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:17.145561 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:17.145671 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:17.146247 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.147132 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.147835 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.148262 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.148392 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:17.150440 (+ 2048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:17.150443 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:17.150444 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:17.150454 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:17.151985 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:17.152324 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:17.152327 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:17.152327 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:17.152403 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:17.152602 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:17.152603 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:17.152603 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:17.152848 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:17.153007 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:17.153068 (+ 61us) server_negotiation.cc:299] Negotiation successful
0925 07:25:17.153216 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":99,"thread_start_us":63,"threads_started":1}
I20260925 07:25:17.153750 7532 catalog_manager.cc:1504] Loading token signing keys...
I20260925 07:25:17.159665 7532 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 39542b3be6b7448ab00f36a31d0c7915: Generated new TSK 0
I20260925 07:25:17.159806 7532 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.158.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab.
Entry for principal kudu/127.5.158.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab.
I20260925 07:25:17.185472 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.65:0
--local_ip_for_outbound_sockets=127.5.158.65
--webserver_interface=127.5.158.65
--webserver_port=0
--tserver_master_addrs=127.5.158.126:37263
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab
--principal=kudu/127.5.158.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:34517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:25:17.270812 7547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:25:17.270982 7547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:25:17.271001 7547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:25:17.272003 7547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:25:17.272037 7547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:25:17.272054 7547 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260925 07:25:17.272063 7547 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 07:25:17.272078 7547 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.65
I20260925 07:25:17.273373 7547 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:34517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab
--principal=kudu/127.5.158.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.158.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.158.126:37263
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7547
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.65
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:25:17.273571 7547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:25:17.273773 7547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:25:17.278240 7555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:17.278326 7547 server_base.cc:1034] running on GCE node
W20260925 07:25:17.278206 7552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:17.278219 7553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:17.278666 7547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:25:17.278890 7547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:25:17.280052 7547 hybrid_clock.cc:648] HybridClock initialized: now 1790321117280039 us; error 31 us; skew 500 ppm
Sep 25 07:25:17 dist-test-slave-1drq krb5kdc[7430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321117, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:25:17.282292 7547 init.cc:377] Logged in from keytab as kudu/127.5.158.65@KRBTEST.COM (short username kudu)
I20260925 07:25:17.282657 7547 webserver.cc:460] Webserver started at http://127.5.158.65:43213/ using document root <none> and password file <none>
I20260925 07:25:17.282866 7547 fs_manager.cc:359] Metadata directory not provided
I20260925 07:25:17.282927 7547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:25:17.283054 7547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:25:17.283905 7547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-0/data/instance:
uuid: "9340b3d9a5314d24bdde11845526f19d"
format_stamp: "Formatted at 2026-09-25 07:25:17 on dist-test-slave-1drq"
server_key: "c9500a0a977b1af059254b4664b84022"
server_key_iv: "ead175d59aee1a6f5f2858e8a3faaa77"
server_key_version: "encryptionkey@0"
I20260925 07:25:17.284211 7547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-0/wal/instance:
uuid: "9340b3d9a5314d24bdde11845526f19d"
format_stamp: "Formatted at 2026-09-25 07:25:17 on dist-test-slave-1drq"
server_key: "c9500a0a977b1af059254b4664b84022"
server_key_iv: "ead175d59aee1a6f5f2858e8a3faaa77"
server_key_version: "encryptionkey@0"
I20260925 07:25:17.285367 7547 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 07:25:17.286015 7562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:17.286163 7547 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:25:17.286255 7547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-0/wal
uuid: "9340b3d9a5314d24bdde11845526f19d"
format_stamp: "Formatted at 2026-09-25 07:25:17 on dist-test-slave-1drq"
server_key: "c9500a0a977b1af059254b4664b84022"
server_key_iv: "ead175d59aee1a6f5f2858e8a3faaa77"
server_key_version: "encryptionkey@0"
I20260925 07:25:17.286330 7547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:25:17.303414 7547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:25:17.305001 7547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:25:17.305192 7547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:25:17.305615 7547 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:25:17.305673 7547 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:17.305711 7547 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:25:17.305737 7547 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:17.312009 7547 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.65:42225
I20260925 07:25:17.312077 7677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.65:42225 every 8 connection(s)
Sep 25 07:25:17 dist-test-slave-1drq krb5kdc[7430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321117, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.65@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:25:17.312307 7547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
I20260925 07:25:17.315128 7544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:17.306114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:58545 (local address 127.5.158.126:37263)
0925 07:25:17.306190 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:17.306193 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:17.306354 (+ 161us) server_negotiation.cc:407] Connection header received
0925 07:25:17.306757 (+ 403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:17.306758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:17.306771 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:17.306853 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:17.307623 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.308369 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.309536 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.309874 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.309942 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:17.312795 (+ 2853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:17.312796 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:17.312797 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:17.312803 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:17.313911 (+ 1108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:17.314282 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:17.314283 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:17.314283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:17.314338 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:17.314548 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:17.314549 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:17.314549 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:17.314656 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:17.314726 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:17.314967 (+ 241us) server_negotiation.cc:299] Negotiation successful
0925 07:25:17.315049 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 07:25:17.315255 7571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:17.306052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37263 (local address 127.5.158.65:58545)
0925 07:25:17.306219 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:17.306262 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:17.306670 (+ 408us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:17.306896 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:17.306899 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:17.307283 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:17.307520 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:17.307526 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.308454 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.308454 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:17.309445 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:17.309447 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.309971 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.309972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:17.310127 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:17.311052 (+ 925us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:17.311062 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:17.312699 (+ 1637us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:17.313997 (+ 1298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:17.314003 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:17.314004 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:17.314191 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:17.314385 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:17.314386 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:17.314386 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:17.314486 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:17.314718 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:17.314724 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:17.314848 (+ 124us) client_negotiation.cc:769] Sending connection context
0925 07:25:17.314945 (+ 97us) client_negotiation.cc:240] Negotiation successful
0925 07:25:17.315072 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":89,"threads_started":1}
I20260925 07:25:17.315716 7678 heartbeater.cc:347] Connected to a master server at 127.5.158.126:37263
I20260925 07:25:17.315798 7678 heartbeater.cc:464] Registering TS with master...
I20260925 07:25:17.315973 7678 heartbeater.cc:511] Master 127.5.158.126:37263 requested a full tablet report, sending...
I20260925 07:25:17.316433 7475 ts_manager.cc:194] Registered new tserver with Master: 9340b3d9a5314d24bdde11845526f19d (127.5.158.65:42225)
I20260925 07:25:17.317313 7475 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.65@KRBTEST.COM'} at 127.5.158.65:58545
I20260925 07:25:17.319855 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 7547
I20260925 07:25:17.319936 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-0/wal/instance
I20260925 07:25:17.320094 5753 external_mini_cluster.cc:1428] Setting key e37a2020bd5130da730f616c4e926a08
I20260925 07:25:17.321561 7544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:17.316774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:35461 (local address 127.5.158.126:37263)
0925 07:25:17.316815 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:17.316817 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:17.316916 (+ 99us) server_negotiation.cc:407] Connection header received
0925 07:25:17.316979 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:17.316979 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:17.316986 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:17.317050 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:17.317389 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.318057 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.318657 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.318805 (+ 148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.318849 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:17.319437 (+ 588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:17.319437 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:17.319438 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:17.319442 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:17.320668 (+ 1226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:17.320978 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:17.320979 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:17.320979 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:17.321019 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:17.321205 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:17.321205 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:17.321205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:17.321291 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:17.321335 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:17.321472 (+ 137us) server_negotiation.cc:299] Negotiation successful
0925 07:25:17.321518 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 07:25:17.321707 7684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:17.316713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37263 (local address 127.5.158.65:35461)
0925 07:25:17.316859 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:17.316869 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:17.316926 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:17.317054 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:17.317054 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:17.317255 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:17.317304 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:17.317305 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.318151 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.318151 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:17.318600 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:17.318601 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.318859 (+ 258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.318860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:17.318905 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:17.319169 (+ 264us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:17.319172 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:17.319374 (+ 202us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:17.320732 (+ 1358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:17.320733 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:17.320733 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:17.320916 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:17.321075 (+ 159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:17.321076 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:17.321076 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:17.321149 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:17.321348 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:17.321348 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:17.321389 (+ 41us) client_negotiation.cc:769] Sending connection context
0925 07:25:17.321460 (+ 71us) client_negotiation.cc:240] Negotiation successful
0925 07:25:17.321554 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":97,"threads_started":1}
WARNING: no policy specified for kudu/127.5.158.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab.
Entry for principal kudu/127.5.158.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab.
I20260925 07:25:17.341786 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.66:0
--local_ip_for_outbound_sockets=127.5.158.66
--webserver_interface=127.5.158.66
--webserver_port=0
--tserver_master_addrs=127.5.158.126:37263
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab
--principal=kudu/127.5.158.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:34517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:25:17.414496 7687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:25:17.414659 7687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:25:17.414678 7687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:25:17.415751 7687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:25:17.415791 7687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:25:17.415809 7687 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260925 07:25:17.415884 7687 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 07:25:17.415920 7687 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.66
I20260925 07:25:17.417388 7687 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:34517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab
--principal=kudu/127.5.158.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.158.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.158.126:37263
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7687
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.66
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:25:17.417583 7687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:25:17.417766 7687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:25:17.421722 7695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:17.421741 7693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:17.421732 7692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:17.421934 7687 server_base.cc:1034] running on GCE node
I20260925 07:25:17.422101 7687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:25:17.422257 7687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:25:17.423389 7687 hybrid_clock.cc:648] HybridClock initialized: now 1790321117423376 us; error 30 us; skew 500 ppm
Sep 25 07:25:17 dist-test-slave-1drq krb5kdc[7430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321117, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:25:17.425333 7687 init.cc:377] Logged in from keytab as kudu/127.5.158.66@KRBTEST.COM (short username kudu)
I20260925 07:25:17.425706 7687 webserver.cc:460] Webserver started at http://127.5.158.66:43037/ using document root <none> and password file <none>
I20260925 07:25:17.425916 7687 fs_manager.cc:359] Metadata directory not provided
I20260925 07:25:17.425981 7687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:25:17.426126 7687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:25:17.427052 7687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-1/data/instance:
uuid: "6b563430b1a945acb4bf3ea743cbca17"
format_stamp: "Formatted at 2026-09-25 07:25:17 on dist-test-slave-1drq"
server_key: "ff77c8c3eafd6e1e263febf0de7324a5"
server_key_iv: "1b8fcd0618a8c37b5d908b9b29eb259a"
server_key_version: "encryptionkey@0"
I20260925 07:25:17.427363 7687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-1/wal/instance:
uuid: "6b563430b1a945acb4bf3ea743cbca17"
format_stamp: "Formatted at 2026-09-25 07:25:17 on dist-test-slave-1drq"
server_key: "ff77c8c3eafd6e1e263febf0de7324a5"
server_key_iv: "1b8fcd0618a8c37b5d908b9b29eb259a"
server_key_version: "encryptionkey@0"
I20260925 07:25:17.428522 7687 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 07:25:17.429121 7702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:17.429267 7687 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:17.429347 7687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-1/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-1/wal
uuid: "6b563430b1a945acb4bf3ea743cbca17"
format_stamp: "Formatted at 2026-09-25 07:25:17 on dist-test-slave-1drq"
server_key: "ff77c8c3eafd6e1e263febf0de7324a5"
server_key_iv: "1b8fcd0618a8c37b5d908b9b29eb259a"
server_key_version: "encryptionkey@0"
I20260925 07:25:17.429415 7687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:25:17.444912 7687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:25:17.446472 7687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:25:17.446623 7687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:25:17.447144 7687 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:25:17.447192 7687 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:17.447230 7687 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:25:17.447252 7687 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 07:25:17 dist-test-slave-1drq krb5kdc[7430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321117, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.66@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:25:17.453327 7687 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.66:35403
I20260925 07:25:17.453377 7817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.66:35403 every 8 connection(s)
I20260925 07:25:17.453642 7687 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
I20260925 07:25:17.455554 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 7687
I20260925 07:25:17.455634 7544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:17.447442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:52479 (local address 127.5.158.126:37263)
0925 07:25:17.447490 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:17.447492 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:17.447663 (+ 171us) server_negotiation.cc:407] Connection header received
0925 07:25:17.448001 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:17.448002 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:17.448016 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:17.448083 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:17.448709 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.449484 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.450341 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.450532 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.450606 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:17.453058 (+ 2452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:17.453059 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:17.453060 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:17.453067 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:17.454462 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:17.454806 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:17.454808 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:17.454808 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:17.454861 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:17.455076 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:17.455077 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:17.455078 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:17.455180 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:17.455242 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:17.455491 (+ 249us) server_negotiation.cc:299] Negotiation successful
0925 07:25:17.455558 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 07:25:17.455655 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-1/wal/instance
I20260925 07:25:17.455780 7711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:17.447422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37263 (local address 127.5.158.66:52479)
0925 07:25:17.447549 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:17.447579 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:17.447917 (+ 338us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:17.448133 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:17.448136 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:17.448365 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:17.448598 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:17.448603 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.449569 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.449570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:17.450253 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:17.450254 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.450615 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.450616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:17.450721 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:17.451134 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:17.451146 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:17.452975 (+ 1829us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:17.454545 (+ 1570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:17.454551 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:17.454553 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:17.454731 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:17.454917 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:17.454917 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:17.454917 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:17.455017 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:17.455245 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:17.455251 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:17.455358 (+ 107us) client_negotiation.cc:769] Sending connection context
0925 07:25:17.455469 (+ 111us) client_negotiation.cc:240] Negotiation successful
0925 07:25:17.455593 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":73,"threads_started":1}
I20260925 07:25:17.455816 5753 external_mini_cluster.cc:1428] Setting key d55de2e9c0d744340c15c1daf4590e8f
I20260925 07:25:17.456564 7818 heartbeater.cc:347] Connected to a master server at 127.5.158.126:37263
I20260925 07:25:17.456663 7818 heartbeater.cc:464] Registering TS with master...
I20260925 07:25:17.456825 7818 heartbeater.cc:511] Master 127.5.158.126:37263 requested a full tablet report, sending...
I20260925 07:25:17.457185 7474 ts_manager.cc:194] Registered new tserver with Master: 6b563430b1a945acb4bf3ea743cbca17 (127.5.158.66:35403)
I20260925 07:25:17.457647 7474 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.66@KRBTEST.COM'} at 127.5.158.66:52479
I20260925 07:25:17.462949 7824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:17.457538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37263 (local address 127.5.158.66:50443)
0925 07:25:17.457790 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:17.457800 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:17.457853 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:17.458000 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:17.458001 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:17.458171 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:17.458216 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:17.458217 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.458722 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.458723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:17.459491 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:17.459494 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.459919 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.459919 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:17.459968 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:17.460332 (+ 364us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:17.460335 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:17.460560 (+ 225us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:17.462139 (+ 1579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:17.462139 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:17.462139 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:17.462269 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:17.462447 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:17.462447 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:17.462447 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:17.462487 (+ 40us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:17.462771 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:17.462771 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:17.462798 (+ 27us) client_negotiation.cc:769] Sending connection context
0925 07:25:17.462836 (+ 38us) client_negotiation.cc:240] Negotiation successful
0925 07:25:17.462890 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":70,"threads_started":1}
I20260925 07:25:17.463011 7544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:17.457680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:50443 (local address 127.5.158.126:37263)
0925 07:25:17.457751 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:17.457753 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:17.457847 (+ 94us) server_negotiation.cc:407] Connection header received
0925 07:25:17.457903 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:17.457904 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:17.457915 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:17.457974 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:17.458279 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.458640 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.459566 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.459850 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.459915 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:17.460618 (+ 703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:17.460620 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:17.460620 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:17.460626 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:17.462071 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:17.462333 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:17.462334 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:17.462334 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:17.462387 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:17.462543 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:17.462544 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:17.462544 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:17.462707 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:17.462771 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:17.462860 (+ 89us) server_negotiation.cc:299] Negotiation successful
0925 07:25:17.462923 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
WARNING: no policy specified for kudu/127.5.158.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab.
Entry for principal kudu/127.5.158.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab.
I20260925 07:25:17.479555 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.67:0
--local_ip_for_outbound_sockets=127.5.158.67
--webserver_interface=127.5.158.67
--webserver_port=0
--tserver_master_addrs=127.5.158.126:37263
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab
--principal=kudu/127.5.158.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:34517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:25:17.554317 7827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:25:17.554473 7827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:25:17.554493 7827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:25:17.555397 7827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:25:17.555445 7827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:25:17.555481 7827 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260925 07:25:17.555492 7827 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 07:25:17.555513 7827 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.67
I20260925 07:25:17.556923 7827 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:34517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab
--principal=kudu/127.5.158.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.158.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.158.126:37263
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7827
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.67
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:25:17.557183 7827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:25:17.557377 7827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:25:17.561561 7835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:17.561564 7833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:17.561604 7827 server_base.cc:1034] running on GCE node
W20260925 07:25:17.561602 7832 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:17.561935 7827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:25:17.562119 7827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:25:17.563280 7827 hybrid_clock.cc:648] HybridClock initialized: now 1790321117563270 us; error 40 us; skew 500 ppm
Sep 25 07:25:17 dist-test-slave-1drq krb5kdc[7430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321117, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:25:17.565330 7827 init.cc:377] Logged in from keytab as kudu/127.5.158.67@KRBTEST.COM (short username kudu)
I20260925 07:25:17.565691 7827 webserver.cc:460] Webserver started at http://127.5.158.67:45577/ using document root <none> and password file <none>
I20260925 07:25:17.565907 7827 fs_manager.cc:359] Metadata directory not provided
I20260925 07:25:17.565972 7827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:25:17.566114 7827 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:25:17.567065 7827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-2/data/instance:
uuid: "a980bf644fe1424688aed206142611df"
format_stamp: "Formatted at 2026-09-25 07:25:17 on dist-test-slave-1drq"
server_key: "38de38d5e8b51d56b4ee660829158746"
server_key_iv: "c44931762994a48d5fc76972e5cf169b"
server_key_version: "encryptionkey@0"
I20260925 07:25:17.567391 7827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-2/wal/instance:
uuid: "a980bf644fe1424688aed206142611df"
format_stamp: "Formatted at 2026-09-25 07:25:17 on dist-test-slave-1drq"
server_key: "38de38d5e8b51d56b4ee660829158746"
server_key_iv: "c44931762994a48d5fc76972e5cf169b"
server_key_version: "encryptionkey@0"
I20260925 07:25:17.568492 7827 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 07:25:17.569170 7842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:17.569326 7827 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:17.569406 7827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-2/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-2/wal
uuid: "a980bf644fe1424688aed206142611df"
format_stamp: "Formatted at 2026-09-25 07:25:17 on dist-test-slave-1drq"
server_key: "38de38d5e8b51d56b4ee660829158746"
server_key_iv: "c44931762994a48d5fc76972e5cf169b"
server_key_version: "encryptionkey@0"
I20260925 07:25:17.569496 7827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:25:17.586201 7827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:25:17.588296 7827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:25:17.588433 7827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:25:17.588867 7827 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:25:17.588924 7827 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:17.588963 7827 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:25:17.589010 7827 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 07:25:17 dist-test-slave-1drq krb5kdc[7430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321117, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.67@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:25:17.595474 7827 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.67:34023
I20260925 07:25:17.595803 7827 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
I20260925 07:25:17.595775 7957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.67:34023 every 8 connection(s)
I20260925 07:25:17.598268 7544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:17.589392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:50247 (local address 127.5.158.126:37263)
0925 07:25:17.589462 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:17.589464 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:17.589696 (+ 232us) server_negotiation.cc:407] Connection header received
0925 07:25:17.590260 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:17.590262 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:17.590276 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:17.590341 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:17.591189 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.591663 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.592633 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.592828 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.592917 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:17.595282 (+ 2365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:17.595284 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:17.595284 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:17.595291 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:17.597157 (+ 1866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:17.597484 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:17.597485 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:17.597485 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:17.597535 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:17.597795 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:17.597795 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:17.597796 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:17.597891 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:17.597948 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:17.598131 (+ 183us) server_negotiation.cc:299] Negotiation successful
0925 07:25:17.598205 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 07:25:17.598366 7851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:17.589368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37263 (local address 127.5.158.67:50247)
0925 07:25:17.589554 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:17.589594 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:17.590138 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:17.590404 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:17.590409 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:17.590786 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:17.591091 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:17.591098 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.591791 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.591792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:17.592531 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:17.592534 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.592927 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.592928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:17.593041 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:17.593496 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:17.593503 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:17.595158 (+ 1655us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:17.597233 (+ 2075us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:17.597236 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:17.597238 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:17.597420 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:17.597593 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:17.597594 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:17.597595 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:17.597698 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:17.597941 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:17.597943 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:17.598028 (+ 85us) client_negotiation.cc:769] Sending connection context
0925 07:25:17.598101 (+ 73us) client_negotiation.cc:240] Negotiation successful
0925 07:25:17.598209 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":95,"threads_started":1}
I20260925 07:25:17.598815 7958 heartbeater.cc:347] Connected to a master server at 127.5.158.126:37263
I20260925 07:25:17.598892 7958 heartbeater.cc:464] Registering TS with master...
I20260925 07:25:17.599090 7958 heartbeater.cc:511] Master 127.5.158.126:37263 requested a full tablet report, sending...
I20260925 07:25:17.599447 7474 ts_manager.cc:194] Registered new tserver with Master: a980bf644fe1424688aed206142611df (127.5.158.67:34023)
I20260925 07:25:17.599956 7474 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.67@KRBTEST.COM'} at 127.5.158.67:50247
I20260925 07:25:17.603587 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 7827
I20260925 07:25:17.603703 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-2/wal/instance
I20260925 07:25:17.603893 5753 external_mini_cluster.cc:1428] Setting key 12f412ffc29f377c9ec44c22033fad6c
I20260925 07:25:17.604343 7544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:17.599791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:50319 (local address 127.5.158.126:37263)
0925 07:25:17.599832 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:17.599833 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:17.599885 (+ 52us) server_negotiation.cc:407] Connection header received
0925 07:25:17.599948 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:17.599948 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:17.599956 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:17.600003 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:17.600335 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.600739 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.601419 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.601615 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.601677 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:17.602309 (+ 632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:17.602310 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:17.602311 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:17.602317 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:17.603324 (+ 1007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:17.603638 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:17.603639 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:17.603639 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:17.603688 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:17.603855 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:17.603856 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:17.603856 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:17.603947 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:17.604015 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:17.604189 (+ 174us) server_negotiation.cc:299] Negotiation successful
0925 07:25:17.604260 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 07:25:17.604414 7964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:17.599687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37263 (local address 127.5.158.67:50319)
0925 07:25:17.599824 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:17.599834 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:17.599888 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:17.600010 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:17.600011 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:17.600192 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:17.600265 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:17.600266 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.600825 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.600826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:17.601339 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:17.601340 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.601679 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.601680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:17.601749 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:17.602031 (+ 282us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:17.602034 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:17.602236 (+ 202us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:17.603408 (+ 1172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:17.603409 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:17.603409 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:17.603560 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:17.603753 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:17.603754 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:17.603755 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:17.603805 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:17.604044 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:17.604045 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:17.604091 (+ 46us) client_negotiation.cc:769] Sending connection context
0925 07:25:17.604151 (+ 60us) client_negotiation.cc:240] Negotiation successful
0925 07:25:17.604319 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":92,"threads_started":1}
I20260925 07:25:17.604621 5753 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 25 07:25:17 dist-test-slave-1drq krb5kdc[7430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321117, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 07:25:17 dist-test-slave-1drq krb5kdc[7430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321117, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:25:17.624908 7544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:17.619310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45954 (local address 127.5.158.126:37263)
0925 07:25:17.619344 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:17.619345 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:17.619350 (+ 5us) server_negotiation.cc:407] Connection header received
0925 07:25:17.619365 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:17.619366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:17.619372 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:17.619435 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:17.619980 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.620379 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.621041 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.621258 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.621333 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:17.623161 (+ 1828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:17.623162 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:17.623162 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:17.623167 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:17.624136 (+ 969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:17.624406 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:17.624407 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:17.624407 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:17.624447 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:17.624577 (+ 130us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:17.624578 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:17.624578 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:17.624647 (+ 69us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:17.624705 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:17.624787 (+ 82us) server_negotiation.cc:299] Negotiation successful
0925 07:25:17.624833 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 07:25:17.626194 7474 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45954:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 07:25:17.627655 7474 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 07:25:17.633432 7973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:17.629528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.67:34023 (local address 127.0.0.1:60872)
0925 07:25:17.629941 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:17.629951 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:17.630040 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:17.630329 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:17.630330 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:17.630332 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:25:17.630360 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:17.630361 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.631512 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.631513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:17.632240 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:17.632242 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.633160 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.633161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:17.633243 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:17.633253 (+ 10us) client_negotiation.cc:769] Sending connection context
0925 07:25:17.633313 (+ 60us) client_negotiation.cc:240] Negotiation successful
0925 07:25:17.633354 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":395,"mutex_wait_us":2,"thread_start_us":47,"threads_started":1}
I20260925 07:25:17.633443 7974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:17.629618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60872 (local address 127.5.158.67:34023)
0925 07:25:17.629774 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:17.629778 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:17.630010 (+ 232us) server_negotiation.cc:407] Connection header received
0925 07:25:17.630136 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:17.630138 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:17.630155 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:17.630217 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:25:17.630434 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.631426 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.632344 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.633032 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.633146 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:17.633193 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:17.633337 (+ 144us) server_negotiation.cc:299] Negotiation successful
0925 07:25:17.633369 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":81,"threads_started":1}
I20260925 07:25:17.633617 7977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:17.629801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46538 (local address 127.5.158.65:42225)
0925 07:25:17.629958 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:17.629963 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:17.630068 (+ 105us) server_negotiation.cc:407] Connection header received
0925 07:25:17.630128 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:17.630130 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:17.630143 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:17.630210 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:25:17.630362 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.631421 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.632630 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.633167 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.633233 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:17.633283 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:17.633472 (+ 189us) server_negotiation.cc:299] Negotiation successful
0925 07:25:17.633514 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":66,"threads_started":1}
I20260925 07:25:17.633782 7972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:17.629528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.66:35403 (local address 127.0.0.1:47972)
0925 07:25:17.629829 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:17.629881 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:17.629949 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:17.630175 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:17.630178 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:17.630180 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:25:17.630230 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:17.630232 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.631159 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.631160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:17.632543 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:17.632545 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.633435 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.633436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:17.633608 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:17.633613 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 07:25:17.633679 (+ 66us) client_negotiation.cc:240] Negotiation successful
0925 07:25:17.633712 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":44,"threads_started":1}
I20260925 07:25:17.633932 7976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:17.629785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47972 (local address 127.5.158.66:35403)
0925 07:25:17.629947 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:17.629951 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:17.629962 (+ 11us) server_negotiation.cc:407] Connection header received
0925 07:25:17.630026 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:17.630027 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:17.630040 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:17.630108 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:25:17.630300 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.631054 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.632611 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.633284 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.633357 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:17.633403 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:17.633798 (+ 395us) server_negotiation.cc:299] Negotiation successful
0925 07:25:17.633839 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":76,"threads_started":1}
I20260925 07:25:17.633901 7975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:17.629689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:42225 (local address 127.0.0.1:46538)
0925 07:25:17.630007 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:17.630016 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:17.630064 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:17.630223 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:17.630224 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:17.630225 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:25:17.630283 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:17.630285 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.631488 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.631489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:17.632552 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:17.632553 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.633310 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.633311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:17.633353 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:17.633356 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 07:25:17.633394 (+ 38us) client_negotiation.cc:240] Negotiation successful
0925 07:25:17.633420 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"mutex_wait_us":28,"thread_start_us":92,"threads_started":1}
I20260925 07:25:17.634147 7892 tablet_service.cc:1467] Processing CreateTablet for tablet 19a80042ed1b4a90af4eb83c2b20d4e1 (DEFAULT_TABLE table=test-table [id=9cd80b2460ae41388c80edae3c7da815]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:25:17.634445 7892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19a80042ed1b4a90af4eb83c2b20d4e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:25:17.634548 7612 tablet_service.cc:1467] Processing CreateTablet for tablet 19a80042ed1b4a90af4eb83c2b20d4e1 (DEFAULT_TABLE table=test-table [id=9cd80b2460ae41388c80edae3c7da815]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:25:17.634833 7612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19a80042ed1b4a90af4eb83c2b20d4e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:25:17.636662 7978 tablet_bootstrap.cc:492] T 19a80042ed1b4a90af4eb83c2b20d4e1 P a980bf644fe1424688aed206142611df: Bootstrap starting.
I20260925 07:25:17.637495 7752 tablet_service.cc:1467] Processing CreateTablet for tablet 19a80042ed1b4a90af4eb83c2b20d4e1 (DEFAULT_TABLE table=test-table [id=9cd80b2460ae41388c80edae3c7da815]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:25:17.637615 7978 tablet_bootstrap.cc:654] T 19a80042ed1b4a90af4eb83c2b20d4e1 P a980bf644fe1424688aed206142611df: Neither blocks nor log segments found. Creating new log.
I20260925 07:25:17.637745 7979 tablet_bootstrap.cc:492] T 19a80042ed1b4a90af4eb83c2b20d4e1 P 9340b3d9a5314d24bdde11845526f19d: Bootstrap starting.
I20260925 07:25:17.637778 7752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19a80042ed1b4a90af4eb83c2b20d4e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:25:17.637920 7978 log.cc:824] T 19a80042ed1b4a90af4eb83c2b20d4e1 P a980bf644fe1424688aed206142611df: Log is configured to *not* fsync() on all Append() calls
I20260925 07:25:17.638286 7979 tablet_bootstrap.cc:654] T 19a80042ed1b4a90af4eb83c2b20d4e1 P 9340b3d9a5314d24bdde11845526f19d: Neither blocks nor log segments found. Creating new log.
I20260925 07:25:17.638581 7979 log.cc:824] T 19a80042ed1b4a90af4eb83c2b20d4e1 P 9340b3d9a5314d24bdde11845526f19d: Log is configured to *not* fsync() on all Append() calls
I20260925 07:25:17.638693 7978 tablet_bootstrap.cc:492] T 19a80042ed1b4a90af4eb83c2b20d4e1 P a980bf644fe1424688aed206142611df: No bootstrap required, opened a new log
I20260925 07:25:17.638787 7978 ts_tablet_manager.cc:1397] T 19a80042ed1b4a90af4eb83c2b20d4e1 P a980bf644fe1424688aed206142611df: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260925 07:25:17.639313 7979 tablet_bootstrap.cc:492] T 19a80042ed1b4a90af4eb83c2b20d4e1 P 9340b3d9a5314d24bdde11845526f19d: No bootstrap required, opened a new log
I20260925 07:25:17.639426 7979 ts_tablet_manager.cc:1397] T 19a80042ed1b4a90af4eb83c2b20d4e1 P 9340b3d9a5314d24bdde11845526f19d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 07:25:17.640357 7982 tablet_bootstrap.cc:492] T 19a80042ed1b4a90af4eb83c2b20d4e1 P 6b563430b1a945acb4bf3ea743cbca17: Bootstrap starting.
I20260925 07:25:17.640892 7978 raft_consensus.cc:348] T 19a80042ed1b4a90af4eb83c2b20d4e1 P a980bf644fe1424688aed206142611df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } } peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } }
I20260925 07:25:17.641037 7978 raft_consensus.cc:374] T 19a80042ed1b4a90af4eb83c2b20d4e1 P a980bf644fe1424688aed206142611df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:25:17.641067 7978 raft_consensus.cc:729] T 19a80042ed1b4a90af4eb83c2b20d4e1 P a980bf644fe1424688aed206142611df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a980bf644fe1424688aed206142611df, State: Initialized, Role: FOLLOWER
I20260925 07:25:17.641179 7982 tablet_bootstrap.cc:654] T 19a80042ed1b4a90af4eb83c2b20d4e1 P 6b563430b1a945acb4bf3ea743cbca17: Neither blocks nor log segments found. Creating new log.
I20260925 07:25:17.641219 7978 consensus_queue.cc:260] T 19a80042ed1b4a90af4eb83c2b20d4e1 P a980bf644fe1424688aed206142611df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } } peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } }
I20260925 07:25:17.641304 7979 raft_consensus.cc:348] T 19a80042ed1b4a90af4eb83c2b20d4e1 P 9340b3d9a5314d24bdde11845526f19d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } } peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } }
I20260925 07:25:17.641470 7958 heartbeater.cc:503] Master 127.5.158.126:37263 was elected leader, sending a full tablet report...
I20260925 07:25:17.641474 7979 raft_consensus.cc:374] T 19a80042ed1b4a90af4eb83c2b20d4e1 P 9340b3d9a5314d24bdde11845526f19d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:25:17.641506 7979 raft_consensus.cc:729] T 19a80042ed1b4a90af4eb83c2b20d4e1 P 9340b3d9a5314d24bdde11845526f19d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9340b3d9a5314d24bdde11845526f19d, State: Initialized, Role: FOLLOWER
I20260925 07:25:17.641528 7982 log.cc:824] T 19a80042ed1b4a90af4eb83c2b20d4e1 P 6b563430b1a945acb4bf3ea743cbca17: Log is configured to *not* fsync() on all Append() calls
I20260925 07:25:17.641600 7979 consensus_queue.cc:260] T 19a80042ed1b4a90af4eb83c2b20d4e1 P 9340b3d9a5314d24bdde11845526f19d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } } peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } }
I20260925 07:25:17.641801 7979 ts_tablet_manager.cc:1428] T 19a80042ed1b4a90af4eb83c2b20d4e1 P 9340b3d9a5314d24bdde11845526f19d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 07:25:17.641896 7678 heartbeater.cc:503] Master 127.5.158.126:37263 was elected leader, sending a full tablet report...
I20260925 07:25:17.641752 7978 ts_tablet_manager.cc:1428] T 19a80042ed1b4a90af4eb83c2b20d4e1 P a980bf644fe1424688aed206142611df: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260925 07:25:17.642741 7982 tablet_bootstrap.cc:492] T 19a80042ed1b4a90af4eb83c2b20d4e1 P 6b563430b1a945acb4bf3ea743cbca17: No bootstrap required, opened a new log
I20260925 07:25:17.642791 7982 ts_tablet_manager.cc:1397] T 19a80042ed1b4a90af4eb83c2b20d4e1 P 6b563430b1a945acb4bf3ea743cbca17: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260925 07:25:17.643966 7982 raft_consensus.cc:348] T 19a80042ed1b4a90af4eb83c2b20d4e1 P 6b563430b1a945acb4bf3ea743cbca17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } } peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } }
I20260925 07:25:17.644068 7982 raft_consensus.cc:374] T 19a80042ed1b4a90af4eb83c2b20d4e1 P 6b563430b1a945acb4bf3ea743cbca17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:25:17.644086 7982 raft_consensus.cc:729] T 19a80042ed1b4a90af4eb83c2b20d4e1 P 6b563430b1a945acb4bf3ea743cbca17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b563430b1a945acb4bf3ea743cbca17, State: Initialized, Role: FOLLOWER
I20260925 07:25:17.644176 7982 consensus_queue.cc:260] T 19a80042ed1b4a90af4eb83c2b20d4e1 P 6b563430b1a945acb4bf3ea743cbca17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } } peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } }
I20260925 07:25:17.644379 7982 ts_tablet_manager.cc:1428] T 19a80042ed1b4a90af4eb83c2b20d4e1 P 6b563430b1a945acb4bf3ea743cbca17: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260925 07:25:17.644450 7818 heartbeater.cc:503] Master 127.5.158.126:37263 was elected leader, sending a full tablet report...
W20260925 07:25:17.704140 7819 tablet.cc:2369] T 19a80042ed1b4a90af4eb83c2b20d4e1 P 6b563430b1a945acb4bf3ea743cbca17: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 07:25:17.795849 7986 raft_consensus.cc:482] T 19a80042ed1b4a90af4eb83c2b20d4e1 P 6b563430b1a945acb4bf3ea743cbca17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 07:25:17.795969 7986 raft_consensus.cc:504] T 19a80042ed1b4a90af4eb83c2b20d4e1 P 6b563430b1a945acb4bf3ea743cbca17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } } peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } }
I20260925 07:25:17.796365 7986 leader_election.cc:288] T 19a80042ed1b4a90af4eb83c2b20d4e1 P 6b563430b1a945acb4bf3ea743cbca17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a980bf644fe1424688aed206142611df (127.5.158.67:34023), 9340b3d9a5314d24bdde11845526f19d (127.5.158.65:42225)
I20260925 07:25:17.798944 7987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:17.796527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.67:34023 (local address 127.5.158.66:59731)
0925 07:25:17.796714 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:17.796725 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:17.796783 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:17.796937 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:17.796938 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:17.796940 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:25:17.797018 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:17.797020 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.797633 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.797634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:17.798251 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:17.798252 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.798713 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.798713 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:17.798781 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:17.798791 (+ 10us) client_negotiation.cc:769] Sending connection context
0925 07:25:17.798829 (+ 38us) client_negotiation.cc:240] Negotiation successful
0925 07:25:17.798854 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":44,"thread_start_us":80,"threads_started":1}
I20260925 07:25:17.798962 7974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:17.796596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:59731 (local address 127.5.158.67:34023)
0925 07:25:17.796638 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:17.796640 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:17.796791 (+ 151us) server_negotiation.cc:407] Connection header received
0925 07:25:17.796841 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:17.796842 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:17.796852 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:17.796909 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:25:17.797126 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.797565 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.798331 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.798642 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.798713 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:17.798720 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:17.798848 (+ 128us) server_negotiation.cc:299] Negotiation successful
0925 07:25:17.798895 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 07:25:17.799003 7977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:17.796596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:41953 (local address 127.5.158.65:42225)
0925 07:25:17.796645 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:17.796647 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:17.796657 (+ 10us) server_negotiation.cc:407] Connection header received
0925 07:25:17.796686 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:17.796686 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:17.796696 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:17.796763 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:25:17.796956 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.797465 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.798256 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.798634 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.798710 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:17.798717 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:17.798906 (+ 189us) server_negotiation.cc:299] Negotiation successful
0925 07:25:17.798938 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260925 07:25:17.799029 7711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:17.796527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:42225 (local address 127.5.158.66:41953)
0925 07:25:17.796568 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:17.796580 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:17.796619 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:17.796805 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:17.796806 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:17.796809 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:25:17.796887 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:17.796889 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.797527 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.797528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:17.798179 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:17.798181 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.798753 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.798754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:17.798836 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:17.798840 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 07:25:17.798888 (+ 48us) client_negotiation.cc:240] Negotiation successful
0925 07:25:17.798920 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260925 07:25:17.799434 7912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19a80042ed1b4a90af4eb83c2b20d4e1" candidate_uuid: "6b563430b1a945acb4bf3ea743cbca17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a980bf644fe1424688aed206142611df" is_pre_election: true
I20260925 07:25:17.799437 7632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19a80042ed1b4a90af4eb83c2b20d4e1" candidate_uuid: "6b563430b1a945acb4bf3ea743cbca17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9340b3d9a5314d24bdde11845526f19d" is_pre_election: true
I20260925 07:25:17.799576 7912 raft_consensus.cc:2454] T 19a80042ed1b4a90af4eb83c2b20d4e1 P a980bf644fe1424688aed206142611df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b563430b1a945acb4bf3ea743cbca17 in term 0.
I20260925 07:25:17.799585 7632 raft_consensus.cc:2454] T 19a80042ed1b4a90af4eb83c2b20d4e1 P 9340b3d9a5314d24bdde11845526f19d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b563430b1a945acb4bf3ea743cbca17 in term 0.
I20260925 07:25:17.799758 7705 leader_election.cc:302] T 19a80042ed1b4a90af4eb83c2b20d4e1 P 6b563430b1a945acb4bf3ea743cbca17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b563430b1a945acb4bf3ea743cbca17, 9340b3d9a5314d24bdde11845526f19d; no voters:
I20260925 07:25:17.799898 7986 raft_consensus.cc:2781] T 19a80042ed1b4a90af4eb83c2b20d4e1 P 6b563430b1a945acb4bf3ea743cbca17 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 07:25:17.799957 7986 raft_consensus.cc:482] T 19a80042ed1b4a90af4eb83c2b20d4e1 P 6b563430b1a945acb4bf3ea743cbca17 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 07:25:17.799973 7986 raft_consensus.cc:3037] T 19a80042ed1b4a90af4eb83c2b20d4e1 P 6b563430b1a945acb4bf3ea743cbca17 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:25:17.800798 7986 raft_consensus.cc:504] T 19a80042ed1b4a90af4eb83c2b20d4e1 P 6b563430b1a945acb4bf3ea743cbca17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } } peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } }
I20260925 07:25:17.800918 7986 leader_election.cc:288] T 19a80042ed1b4a90af4eb83c2b20d4e1 P 6b563430b1a945acb4bf3ea743cbca17 [CANDIDATE]: Term 1 election: Requested vote from peers a980bf644fe1424688aed206142611df (127.5.158.67:34023), 9340b3d9a5314d24bdde11845526f19d (127.5.158.65:42225)
I20260925 07:25:17.801059 7912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19a80042ed1b4a90af4eb83c2b20d4e1" candidate_uuid: "6b563430b1a945acb4bf3ea743cbca17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a980bf644fe1424688aed206142611df"
I20260925 07:25:17.801059 7632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19a80042ed1b4a90af4eb83c2b20d4e1" candidate_uuid: "6b563430b1a945acb4bf3ea743cbca17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9340b3d9a5314d24bdde11845526f19d"
I20260925 07:25:17.801160 7632 raft_consensus.cc:3037] T 19a80042ed1b4a90af4eb83c2b20d4e1 P 9340b3d9a5314d24bdde11845526f19d [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:25:17.801162 7912 raft_consensus.cc:3037] T 19a80042ed1b4a90af4eb83c2b20d4e1 P a980bf644fe1424688aed206142611df [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:25:17.801962 7632 raft_consensus.cc:2454] T 19a80042ed1b4a90af4eb83c2b20d4e1 P 9340b3d9a5314d24bdde11845526f19d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b563430b1a945acb4bf3ea743cbca17 in term 1.
I20260925 07:25:17.801963 7912 raft_consensus.cc:2454] T 19a80042ed1b4a90af4eb83c2b20d4e1 P a980bf644fe1424688aed206142611df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b563430b1a945acb4bf3ea743cbca17 in term 1.
I20260925 07:25:17.802119 7706 leader_election.cc:302] T 19a80042ed1b4a90af4eb83c2b20d4e1 P 6b563430b1a945acb4bf3ea743cbca17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b563430b1a945acb4bf3ea743cbca17, a980bf644fe1424688aed206142611df; no voters:
I20260925 07:25:17.802239 7986 raft_consensus.cc:2781] T 19a80042ed1b4a90af4eb83c2b20d4e1 P 6b563430b1a945acb4bf3ea743cbca17 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:25:17.802350 7986 raft_consensus.cc:686] T 19a80042ed1b4a90af4eb83c2b20d4e1 P 6b563430b1a945acb4bf3ea743cbca17 [term 1 LEADER]: Becoming Leader. State: Replica: 6b563430b1a945acb4bf3ea743cbca17, State: Running, Role: LEADER
I20260925 07:25:17.802407 7986 consensus_queue.cc:237] T 19a80042ed1b4a90af4eb83c2b20d4e1 P 6b563430b1a945acb4bf3ea743cbca17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } } peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } }
I20260925 07:25:17.803107 7473 catalog_manager.cc:5495] T 19a80042ed1b4a90af4eb83c2b20d4e1 P 6b563430b1a945acb4bf3ea743cbca17 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b563430b1a945acb4bf3ea743cbca17 (127.5.158.66). New cstate: current_term: 1 leader_uuid: "6b563430b1a945acb4bf3ea743cbca17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } health_report { overall_health: UNKNOWN } } }
W20260925 07:25:17.812978 7679 tablet.cc:2369] T 19a80042ed1b4a90af4eb83c2b20d4e1 P 9340b3d9a5314d24bdde11845526f19d: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 25 07:25:17 dist-test-slave-1drq krb5kdc[7430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321117, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.126@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:25:17.831377 7544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:17.825777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45958 (local address 127.5.158.126:37263)
0925 07:25:17.825860 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:17.825862 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:17.825931 (+ 69us) server_negotiation.cc:407] Connection header received
0925 07:25:17.825969 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:17.825970 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:17.825978 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:17.826024 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:17.826425 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.826877 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.827433 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.827574 (+ 141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.827613 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:17.829526 (+ 1913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:17.829527 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:17.829527 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:17.829532 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:17.830544 (+ 1012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:17.830818 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:17.830819 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:17.830819 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:17.830888 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:17.831015 (+ 127us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:17.831015 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:17.831015 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:17.831088 (+ 73us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:17.831125 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:17.831242 (+ 117us) server_negotiation.cc:299] Negotiation successful
0925 07:25:17.831327 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260925 07:25:17.831504 7995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:17.825710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37263 (local address 127.0.0.1:45958)
0925 07:25:17.825867 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:17.825876 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:17.825931 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:17.826037 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:17.826038 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:17.826316 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:17.826364 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:17.826366 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.826939 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.826940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:17.827380 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:17.827380 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.827624 (+ 244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.827625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:17.827664 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:17.828149 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:17.828153 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:17.829455 (+ 1302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:17.830602 (+ 1147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:17.830607 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:17.830609 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:17.830764 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:17.830936 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:17.830937 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:17.830937 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:17.830982 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:17.831133 (+ 151us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:17.831139 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:17.831178 (+ 39us) client_negotiation.cc:769] Sending connection context
0925 07:25:17.831228 (+ 50us) client_negotiation.cc:240] Negotiation successful
0925 07:25:17.831370 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":74,"thread_start_us":75,"threads_started":1}
I20260925 07:25:17.833211 7473 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:45958:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260925 07:25:17.833417 7473 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 07:25:17.834612 7612 tablet_service.cc:1467] Processing CreateTablet for tablet 58a443993da047d28d3d4922dd5ab7d7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8c9d33ad83334112916481e85fc135ae]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 07:25:17.834650 7892 tablet_service.cc:1467] Processing CreateTablet for tablet 58a443993da047d28d3d4922dd5ab7d7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8c9d33ad83334112916481e85fc135ae]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 07:25:17.834751 7612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58a443993da047d28d3d4922dd5ab7d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:25:17.834724 7752 tablet_service.cc:1467] Processing CreateTablet for tablet 58a443993da047d28d3d4922dd5ab7d7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8c9d33ad83334112916481e85fc135ae]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 07:25:17.834760 7892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58a443993da047d28d3d4922dd5ab7d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:25:17.834824 7752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58a443993da047d28d3d4922dd5ab7d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:25:17.835723 7982 tablet_bootstrap.cc:492] T 58a443993da047d28d3d4922dd5ab7d7 P 6b563430b1a945acb4bf3ea743cbca17: Bootstrap starting.
I20260925 07:25:17.836062 7979 tablet_bootstrap.cc:492] T 58a443993da047d28d3d4922dd5ab7d7 P 9340b3d9a5314d24bdde11845526f19d: Bootstrap starting.
I20260925 07:25:17.836126 7982 tablet_bootstrap.cc:654] T 58a443993da047d28d3d4922dd5ab7d7 P 6b563430b1a945acb4bf3ea743cbca17: Neither blocks nor log segments found. Creating new log.
I20260925 07:25:17.836455 7979 tablet_bootstrap.cc:654] T 58a443993da047d28d3d4922dd5ab7d7 P 9340b3d9a5314d24bdde11845526f19d: Neither blocks nor log segments found. Creating new log.
I20260925 07:25:17.836759 7978 tablet_bootstrap.cc:492] T 58a443993da047d28d3d4922dd5ab7d7 P a980bf644fe1424688aed206142611df: Bootstrap starting.
I20260925 07:25:17.836844 7982 tablet_bootstrap.cc:492] T 58a443993da047d28d3d4922dd5ab7d7 P 6b563430b1a945acb4bf3ea743cbca17: No bootstrap required, opened a new log
I20260925 07:25:17.836890 7982 ts_tablet_manager.cc:1397] T 58a443993da047d28d3d4922dd5ab7d7 P 6b563430b1a945acb4bf3ea743cbca17: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260925 07:25:17.837044 7982 raft_consensus.cc:348] T 58a443993da047d28d3d4922dd5ab7d7 P 6b563430b1a945acb4bf3ea743cbca17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } } peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } }
I20260925 07:25:17.837163 7982 raft_consensus.cc:374] T 58a443993da047d28d3d4922dd5ab7d7 P 6b563430b1a945acb4bf3ea743cbca17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:25:17.837185 7982 raft_consensus.cc:729] T 58a443993da047d28d3d4922dd5ab7d7 P 6b563430b1a945acb4bf3ea743cbca17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b563430b1a945acb4bf3ea743cbca17, State: Initialized, Role: FOLLOWER
I20260925 07:25:17.837225 7982 consensus_queue.cc:260] T 58a443993da047d28d3d4922dd5ab7d7 P 6b563430b1a945acb4bf3ea743cbca17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } } peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } }
I20260925 07:25:17.837323 7982 ts_tablet_manager.cc:1428] T 58a443993da047d28d3d4922dd5ab7d7 P 6b563430b1a945acb4bf3ea743cbca17: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:17.837344 7978 tablet_bootstrap.cc:654] T 58a443993da047d28d3d4922dd5ab7d7 P a980bf644fe1424688aed206142611df: Neither blocks nor log segments found. Creating new log.
I20260925 07:25:17.837357 7986 tablet_replica.cc:439] T 58a443993da047d28d3d4922dd5ab7d7 P 6b563430b1a945acb4bf3ea743cbca17: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } } peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } } }
I20260925 07:25:17.837443 7986 tablet_replica.cc:442] T 58a443993da047d28d3d4922dd5ab7d7 P 6b563430b1a945acb4bf3ea743cbca17: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 07:25:17.837813 7979 tablet_bootstrap.cc:492] T 58a443993da047d28d3d4922dd5ab7d7 P 9340b3d9a5314d24bdde11845526f19d: No bootstrap required, opened a new log
I20260925 07:25:17.837859 7979 ts_tablet_manager.cc:1397] T 58a443993da047d28d3d4922dd5ab7d7 P 9340b3d9a5314d24bdde11845526f19d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 07:25:17.837908 7978 tablet_bootstrap.cc:492] T 58a443993da047d28d3d4922dd5ab7d7 P a980bf644fe1424688aed206142611df: No bootstrap required, opened a new log
I20260925 07:25:17.837944 7978 ts_tablet_manager.cc:1397] T 58a443993da047d28d3d4922dd5ab7d7 P a980bf644fe1424688aed206142611df: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 07:25:17.838001 7979 raft_consensus.cc:348] T 58a443993da047d28d3d4922dd5ab7d7 P 9340b3d9a5314d24bdde11845526f19d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } } peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } }
I20260925 07:25:17.838057 7979 raft_consensus.cc:374] T 58a443993da047d28d3d4922dd5ab7d7 P 9340b3d9a5314d24bdde11845526f19d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:25:17.838079 7979 raft_consensus.cc:729] T 58a443993da047d28d3d4922dd5ab7d7 P 9340b3d9a5314d24bdde11845526f19d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9340b3d9a5314d24bdde11845526f19d, State: Initialized, Role: FOLLOWER
I20260925 07:25:17.838105 7978 raft_consensus.cc:348] T 58a443993da047d28d3d4922dd5ab7d7 P a980bf644fe1424688aed206142611df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } } peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } }
I20260925 07:25:17.838171 7978 raft_consensus.cc:374] T 58a443993da047d28d3d4922dd5ab7d7 P a980bf644fe1424688aed206142611df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:25:17.838145 7979 consensus_queue.cc:260] T 58a443993da047d28d3d4922dd5ab7d7 P 9340b3d9a5314d24bdde11845526f19d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } } peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } }
I20260925 07:25:17.838187 7978 raft_consensus.cc:729] T 58a443993da047d28d3d4922dd5ab7d7 P a980bf644fe1424688aed206142611df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a980bf644fe1424688aed206142611df, State: Initialized, Role: FOLLOWER
I20260925 07:25:17.838225 7978 consensus_queue.cc:260] T 58a443993da047d28d3d4922dd5ab7d7 P a980bf644fe1424688aed206142611df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } } peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } }
I20260925 07:25:17.838245 7979 ts_tablet_manager.cc:1428] T 58a443993da047d28d3d4922dd5ab7d7 P 9340b3d9a5314d24bdde11845526f19d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:17.838263 7985 tablet_replica.cc:439] T 58a443993da047d28d3d4922dd5ab7d7 P 9340b3d9a5314d24bdde11845526f19d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } } peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } } }
I20260925 07:25:17.838317 7978 ts_tablet_manager.cc:1428] T 58a443993da047d28d3d4922dd5ab7d7 P a980bf644fe1424688aed206142611df: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:17.838330 7985 tablet_replica.cc:442] T 58a443993da047d28d3d4922dd5ab7d7 P 9340b3d9a5314d24bdde11845526f19d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 07:25:17.838433 7983 tablet_replica.cc:439] T 58a443993da047d28d3d4922dd5ab7d7 P a980bf644fe1424688aed206142611df: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } } peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } } }
I20260925 07:25:17.838526 7983 tablet_replica.cc:442] T 58a443993da047d28d3d4922dd5ab7d7 P a980bf644fe1424688aed206142611df: This TxnStatusTablet replica's current role is: FOLLOWER
W20260925 07:25:17.846657 7959 tablet.cc:2369] T 19a80042ed1b4a90af4eb83c2b20d4e1 P a980bf644fe1424688aed206142611df: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 07:25:17.968652 7985 raft_consensus.cc:482] T 58a443993da047d28d3d4922dd5ab7d7 P 9340b3d9a5314d24bdde11845526f19d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 07:25:17.968737 7985 raft_consensus.cc:504] T 58a443993da047d28d3d4922dd5ab7d7 P 9340b3d9a5314d24bdde11845526f19d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } } peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } }
I20260925 07:25:17.969102 7985 leader_election.cc:288] T 58a443993da047d28d3d4922dd5ab7d7 P 9340b3d9a5314d24bdde11845526f19d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b563430b1a945acb4bf3ea743cbca17 (127.5.158.66:35403), a980bf644fe1424688aed206142611df (127.5.158.67:34023)
I20260925 07:25:17.971725 7999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:17.969241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.66:35403 (local address 127.5.158.65:44905)
0925 07:25:17.969364 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:17.969372 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:17.969446 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:17.969605 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:17.969606 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:17.969607 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:25:17.969690 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:17.969692 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.970219 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.970219 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:17.970920 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:17.970922 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.971555 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.971556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:17.971601 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:17.971604 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 07:25:17.971640 (+ 36us) client_negotiation.cc:240] Negotiation successful
0925 07:25:17.971664 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"mutex_wait_us":26,"thread_start_us":74,"threads_started":1}
I20260925 07:25:17.971761 7976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:17.969321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:44905 (local address 127.5.158.66:35403)
0925 07:25:17.969362 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:17.969364 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:17.969410 (+ 46us) server_negotiation.cc:407] Connection header received
0925 07:25:17.969536 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:17.969537 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:17.969547 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:17.969590 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:25:17.969756 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.970132 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.971006 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.971463 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.971524 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:17.971532 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:17.971661 (+ 129us) server_negotiation.cc:299] Negotiation successful
0925 07:25:17.971697 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 07:25:17.971972 7974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:17.969301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:48475 (local address 127.5.158.67:34023)
0925 07:25:17.969345 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:17.969347 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:17.969439 (+ 92us) server_negotiation.cc:407] Connection header received
0925 07:25:17.969484 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:17.969485 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:17.969493 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:17.969546 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:25:17.969673 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.970116 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.971008 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.971307 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.971411 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:17.971421 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:17.971857 (+ 436us) server_negotiation.cc:299] Negotiation successful
0925 07:25:17.971913 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 07:25:17.972034 7771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58a443993da047d28d3d4922dd5ab7d7" candidate_uuid: "9340b3d9a5314d24bdde11845526f19d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b563430b1a945acb4bf3ea743cbca17" is_pre_election: true
I20260925 07:25:17.972141 7771 raft_consensus.cc:2454] T 58a443993da047d28d3d4922dd5ab7d7 P 6b563430b1a945acb4bf3ea743cbca17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9340b3d9a5314d24bdde11845526f19d in term 0.
I20260925 07:25:17.972210 8000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:17.969260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.67:34023 (local address 127.5.158.65:48475)
0925 07:25:17.969369 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:17.969377 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:17.969434 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:17.969556 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:17.969557 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:17.969559 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:25:17.969612 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:17.969614 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.970185 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.970186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:17.970922 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:17.970924 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.971676 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.971677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:17.971730 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:17.971735 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 07:25:17.972095 (+ 360us) client_negotiation.cc:240] Negotiation successful
0925 07:25:17.972135 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"mutex_wait_us":42,"thread_start_us":49,"threads_started":1}
I20260925 07:25:17.972311 7565 leader_election.cc:302] T 58a443993da047d28d3d4922dd5ab7d7 P 9340b3d9a5314d24bdde11845526f19d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b563430b1a945acb4bf3ea743cbca17, 9340b3d9a5314d24bdde11845526f19d; no voters:
I20260925 07:25:17.972442 7985 raft_consensus.cc:2781] T 58a443993da047d28d3d4922dd5ab7d7 P 9340b3d9a5314d24bdde11845526f19d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 07:25:17.972491 7985 raft_consensus.cc:482] T 58a443993da047d28d3d4922dd5ab7d7 P 9340b3d9a5314d24bdde11845526f19d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 07:25:17.972518 7985 raft_consensus.cc:3037] T 58a443993da047d28d3d4922dd5ab7d7 P 9340b3d9a5314d24bdde11845526f19d [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:25:17.972517 7912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58a443993da047d28d3d4922dd5ab7d7" candidate_uuid: "9340b3d9a5314d24bdde11845526f19d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a980bf644fe1424688aed206142611df" is_pre_election: true
I20260925 07:25:17.972592 7912 raft_consensus.cc:2454] T 58a443993da047d28d3d4922dd5ab7d7 P a980bf644fe1424688aed206142611df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9340b3d9a5314d24bdde11845526f19d in term 0.
I20260925 07:25:17.973227 7985 raft_consensus.cc:504] T 58a443993da047d28d3d4922dd5ab7d7 P 9340b3d9a5314d24bdde11845526f19d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } } peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } }
I20260925 07:25:17.973349 7985 leader_election.cc:288] T 58a443993da047d28d3d4922dd5ab7d7 P 9340b3d9a5314d24bdde11845526f19d [CANDIDATE]: Term 1 election: Requested vote from peers 6b563430b1a945acb4bf3ea743cbca17 (127.5.158.66:35403), a980bf644fe1424688aed206142611df (127.5.158.67:34023)
I20260925 07:25:17.973460 7771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58a443993da047d28d3d4922dd5ab7d7" candidate_uuid: "9340b3d9a5314d24bdde11845526f19d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b563430b1a945acb4bf3ea743cbca17"
I20260925 07:25:17.973548 7771 raft_consensus.cc:3037] T 58a443993da047d28d3d4922dd5ab7d7 P 6b563430b1a945acb4bf3ea743cbca17 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:25:17.973485 7912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58a443993da047d28d3d4922dd5ab7d7" candidate_uuid: "9340b3d9a5314d24bdde11845526f19d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a980bf644fe1424688aed206142611df"
I20260925 07:25:17.973592 7912 raft_consensus.cc:3037] T 58a443993da047d28d3d4922dd5ab7d7 P a980bf644fe1424688aed206142611df [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:25:17.974152 7771 raft_consensus.cc:2454] T 58a443993da047d28d3d4922dd5ab7d7 P 6b563430b1a945acb4bf3ea743cbca17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9340b3d9a5314d24bdde11845526f19d in term 1.
I20260925 07:25:17.974156 7912 raft_consensus.cc:2454] T 58a443993da047d28d3d4922dd5ab7d7 P a980bf644fe1424688aed206142611df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9340b3d9a5314d24bdde11845526f19d in term 1.
I20260925 07:25:17.974323 7565 leader_election.cc:302] T 58a443993da047d28d3d4922dd5ab7d7 P 9340b3d9a5314d24bdde11845526f19d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b563430b1a945acb4bf3ea743cbca17, 9340b3d9a5314d24bdde11845526f19d; no voters:
I20260925 07:25:17.974445 7985 raft_consensus.cc:2781] T 58a443993da047d28d3d4922dd5ab7d7 P 9340b3d9a5314d24bdde11845526f19d [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:25:17.974572 7985 raft_consensus.cc:686] T 58a443993da047d28d3d4922dd5ab7d7 P 9340b3d9a5314d24bdde11845526f19d [term 1 LEADER]: Becoming Leader. State: Replica: 9340b3d9a5314d24bdde11845526f19d, State: Running, Role: LEADER
I20260925 07:25:17.974674 7985 consensus_queue.cc:237] T 58a443993da047d28d3d4922dd5ab7d7 P 9340b3d9a5314d24bdde11845526f19d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } } peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } }
I20260925 07:25:17.975093 8001 tablet_replica.cc:439] T 58a443993da047d28d3d4922dd5ab7d7 P 9340b3d9a5314d24bdde11845526f19d: TxnStatusTablet state changed. Reason: New leader 9340b3d9a5314d24bdde11845526f19d. Latest consensus state: current_term: 1 leader_uuid: "9340b3d9a5314d24bdde11845526f19d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } } peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } } }
I20260925 07:25:17.975169 8001 tablet_replica.cc:442] T 58a443993da047d28d3d4922dd5ab7d7 P 9340b3d9a5314d24bdde11845526f19d: This TxnStatusTablet replica's current role is: LEADER
I20260925 07:25:17.975324 8003 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260925 07:25:17.975441 8003 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260925 07:25:17.975426 7474 catalog_manager.cc:5495] T 58a443993da047d28d3d4922dd5ab7d7 P 9340b3d9a5314d24bdde11845526f19d reported cstate change: term changed from 0 to 1, leader changed from <none> to 9340b3d9a5314d24bdde11845526f19d (127.5.158.65). New cstate: current_term: 1 leader_uuid: "9340b3d9a5314d24bdde11845526f19d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } health_report { overall_health: UNKNOWN } } }
I20260925 07:25:17.982172 7995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:17.979742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:42225 (local address 127.0.0.1:46548)
0925 07:25:17.979804 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:17.979815 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:17.979859 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:17.980034 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:17.980035 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:17.980035 (+ 0us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 07:25:17.980071 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:17.980072 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.980799 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.980800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:17.981338 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:17.981339 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.981745 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.981746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:17.981786 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:17.981800 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 07:25:17.982068 (+ 268us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 07:25:17.982073 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 07:25:17.982097 (+ 24us) client_negotiation.cc:240] Negotiation successful
0925 07:25:17.982127 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260925 07:25:17.982180 7977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:17.979857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46548 (local address 127.5.158.65:42225)
0925 07:25:17.979910 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:17.979912 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:17.979919 (+ 7us) server_negotiation.cc:407] Connection header received
0925 07:25:17.979944 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:17.979945 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:17.979960 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:17.980034 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 07:25:17.980119 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.980735 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.981408 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.981672 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.981745 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:17.981884 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 07:25:17.982009 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 07:25:17.982094 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:17.982109 (+ 15us) server_negotiation.cc:299] Negotiation successful
0925 07:25:17.982125 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260925 07:25:17.983701 7912 raft_consensus.cc:1261] T 58a443993da047d28d3d4922dd5ab7d7 P a980bf644fe1424688aed206142611df [term 1 FOLLOWER]: Refusing update from remote peer 9340b3d9a5314d24bdde11845526f19d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:25:17.983732 7771 raft_consensus.cc:1261] T 58a443993da047d28d3d4922dd5ab7d7 P 6b563430b1a945acb4bf3ea743cbca17 [term 1 FOLLOWER]: Refusing update from remote peer 9340b3d9a5314d24bdde11845526f19d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:25:17.984010 8001 consensus_queue.cc:1035] T 58a443993da047d28d3d4922dd5ab7d7 P 9340b3d9a5314d24bdde11845526f19d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:25:17.984104 7985 consensus_queue.cc:1035] T 58a443993da047d28d3d4922dd5ab7d7 P 9340b3d9a5314d24bdde11845526f19d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:25:17.985038 7986 tablet_replica.cc:439] T 58a443993da047d28d3d4922dd5ab7d7 P 6b563430b1a945acb4bf3ea743cbca17: TxnStatusTablet state changed. Reason: New leader 9340b3d9a5314d24bdde11845526f19d. Latest consensus state: current_term: 1 leader_uuid: "9340b3d9a5314d24bdde11845526f19d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } } peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } } }
I20260925 07:25:17.985137 7986 tablet_replica.cc:442] T 58a443993da047d28d3d4922dd5ab7d7 P 6b563430b1a945acb4bf3ea743cbca17: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 07:25:17.985271 7983 tablet_replica.cc:439] T 58a443993da047d28d3d4922dd5ab7d7 P a980bf644fe1424688aed206142611df: TxnStatusTablet state changed. Reason: New leader 9340b3d9a5314d24bdde11845526f19d. Latest consensus state: current_term: 1 leader_uuid: "9340b3d9a5314d24bdde11845526f19d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } } peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } } }
I20260925 07:25:17.985337 7983 tablet_replica.cc:442] T 58a443993da047d28d3d4922dd5ab7d7 P a980bf644fe1424688aed206142611df: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 07:25:17.986177 8001 tablet_replica.cc:439] T 58a443993da047d28d3d4922dd5ab7d7 P 9340b3d9a5314d24bdde11845526f19d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9340b3d9a5314d24bdde11845526f19d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } } peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } } }
I20260925 07:25:17.986253 8001 tablet_replica.cc:442] T 58a443993da047d28d3d4922dd5ab7d7 P 9340b3d9a5314d24bdde11845526f19d: This TxnStatusTablet replica's current role is: LEADER
I20260925 07:25:17.986290 7986 tablet_replica.cc:439] T 58a443993da047d28d3d4922dd5ab7d7 P 6b563430b1a945acb4bf3ea743cbca17: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9340b3d9a5314d24bdde11845526f19d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } } peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } } }
I20260925 07:25:17.986347 7986 tablet_replica.cc:442] T 58a443993da047d28d3d4922dd5ab7d7 P 6b563430b1a945acb4bf3ea743cbca17: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 07:25:17.986557 8007 mvcc.cc:205] Tried to move back new op lower bound from 7333155299259248640 to 7333155299224748032. Current Snapshot: MvccSnapshot[applied={T|T < 7333155299259248640}]
I20260925 07:25:17.986660 7985 tablet_replica.cc:439] T 58a443993da047d28d3d4922dd5ab7d7 P 9340b3d9a5314d24bdde11845526f19d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9340b3d9a5314d24bdde11845526f19d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } } peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } } }
I20260925 07:25:17.986730 7985 tablet_replica.cc:442] T 58a443993da047d28d3d4922dd5ab7d7 P 9340b3d9a5314d24bdde11845526f19d: This TxnStatusTablet replica's current role is: LEADER
I20260925 07:25:17.987222 7983 tablet_replica.cc:439] T 58a443993da047d28d3d4922dd5ab7d7 P a980bf644fe1424688aed206142611df: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9340b3d9a5314d24bdde11845526f19d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } } peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } } }
I20260925 07:25:17.987300 7983 tablet_replica.cc:442] T 58a443993da047d28d3d4922dd5ab7d7 P a980bf644fe1424688aed206142611df: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 07:25:17.988077 8008 mvcc.cc:205] Tried to move back new op lower bound from 7333155299259248640 to 7333155299224748032. Current Snapshot: MvccSnapshot[applied={T|T < 7333155299259248640 or (T in {7333155299259248640})}]
I20260925 07:25:17.990674 7976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:17.988511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47974 (local address 127.5.158.66:35403)
0925 07:25:17.988554 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:17.988555 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:17.988564 (+ 9us) server_negotiation.cc:407] Connection header received
0925 07:25:17.988618 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:17.988619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:17.988633 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:17.988685 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 07:25:17.988803 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.989252 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.990005 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.990178 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.990238 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:17.990410 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 07:25:17.990496 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 07:25:17.990569 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:17.990601 (+ 32us) server_negotiation.cc:299] Negotiation successful
0925 07:25:17.990616 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 07:25:17.994838 7977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:17.992612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:45981 (local address 127.5.158.65:42225)
0925 07:25:17.992646 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:17.992648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:17.992740 (+ 92us) server_negotiation.cc:407] Connection header received
0925 07:25:17.992793 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:17.992794 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:17.992800 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:17.992837 (+ 37us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 07:25:17.992954 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.993367 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.994310 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.994469 (+ 159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.994512 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:17.994687 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 07:25:17.994706 (+ 19us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 07:25:17.994739 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:17.994787 (+ 48us) server_negotiation.cc:299] Negotiation successful
0925 07:25:17.994801 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 07:25:17.994894 8017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:17.992542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:42225 (local address 127.5.158.66:45981)
0925 07:25:17.992686 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:17.992695 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:17.992746 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:17.992856 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:17.992857 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:17.992858 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 07:25:17.992905 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:17.992906 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.993427 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.993428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:17.994236 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:17.994238 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:17.994532 (+ 294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:17.994533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:17.994594 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:17.994606 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 07:25:17.994740 (+ 134us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 07:25:17.994741 (+ 1us) client_negotiation.cc:769] Sending connection context
0925 07:25:17.994784 (+ 43us) client_negotiation.cc:240] Negotiation successful
0925 07:25:17.994818 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":77,"threads_started":1}
I20260925 07:25:17.997218 7632 raft_consensus.cc:1261] T 19a80042ed1b4a90af4eb83c2b20d4e1 P 9340b3d9a5314d24bdde11845526f19d [term 1 FOLLOWER]: Refusing update from remote peer 6b563430b1a945acb4bf3ea743cbca17: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:25:17.997254 7912 raft_consensus.cc:1261] T 19a80042ed1b4a90af4eb83c2b20d4e1 P a980bf644fe1424688aed206142611df [term 1 FOLLOWER]: Refusing update from remote peer 6b563430b1a945acb4bf3ea743cbca17: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:25:17.997457 7988 consensus_queue.cc:1035] T 19a80042ed1b4a90af4eb83c2b20d4e1 P 6b563430b1a945acb4bf3ea743cbca17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:25:17.997548 7986 consensus_queue.cc:1035] T 19a80042ed1b4a90af4eb83c2b20d4e1 P 6b563430b1a945acb4bf3ea743cbca17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:25:17.999442 8006 mvcc.cc:205] Tried to move back new op lower bound from 7333155299314872320 to 7333155298518958080. Current Snapshot: MvccSnapshot[applied={T|T < 7333155299314872320}]
I20260925 07:25:18.004590 7976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:18.002490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:56123 (local address 127.5.158.66:35403)
0925 07:25:18.002519 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:18.002521 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:18.002528 (+ 7us) server_negotiation.cc:407] Connection header received
0925 07:25:18.002584 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:18.002585 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:18.002591 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:18.002629 (+ 38us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 07:25:18.002733 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:18.003084 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:18.004029 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:18.004251 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:18.004295 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:18.004450 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 07:25:18.004468 (+ 18us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 07:25:18.004499 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:18.004544 (+ 45us) server_negotiation.cc:299] Negotiation successful
0925 07:25:18.004556 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260925 07:25:18.004669 8023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:18.002337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.66:35403 (local address 127.5.158.65:56123)
0925 07:25:18.002487 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:18.002496 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:18.002535 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:18.002631 (+ 96us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:18.002632 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:18.002633 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 07:25:18.002676 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:18.002677 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:18.003149 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:18.003151 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:18.003971 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:18.003973 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:18.004312 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:18.004312 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:18.004368 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:18.004380 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 07:25:18.004501 (+ 121us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 07:25:18.004502 (+ 1us) client_negotiation.cc:769] Sending connection context
0925 07:25:18.004543 (+ 41us) client_negotiation.cc:240] Negotiation successful
0925 07:25:18.004573 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":75,"threads_started":1}
I20260925 07:25:18.014576 7474 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45954:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 07:25:18.014660 7474 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45954:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 07:25:18.015293 7474 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 39542b3be6b7448ab00f36a31d0c7915: Sending DeleteTablet for 3 replicas of tablet 19a80042ed1b4a90af4eb83c2b20d4e1
I20260925 07:25:18.015575 7892 tablet_service.cc:1514] Processing DeleteTablet for tablet 19a80042ed1b4a90af4eb83c2b20d4e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:25:18 UTC) from {username='kudu', principal='kudu/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:60872
I20260925 07:25:18.015604 7752 tablet_service.cc:1514] Processing DeleteTablet for tablet 19a80042ed1b4a90af4eb83c2b20d4e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:25:18 UTC) from {username='kudu', principal='kudu/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:47972
I20260925 07:25:18.015780 8027 tablet_replica.cc:330] T 19a80042ed1b4a90af4eb83c2b20d4e1 P a980bf644fe1424688aed206142611df: stopping tablet replica
I20260925 07:25:18.015813 7612 tablet_service.cc:1514] Processing DeleteTablet for tablet 19a80042ed1b4a90af4eb83c2b20d4e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:25:18 UTC) from {username='kudu', principal='kudu/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:46538
I20260925 07:25:18.015898 8027 raft_consensus.cc:2229] T 19a80042ed1b4a90af4eb83c2b20d4e1 P a980bf644fe1424688aed206142611df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 07:25:18.015964 8027 raft_consensus.cc:2258] T 19a80042ed1b4a90af4eb83c2b20d4e1 P a980bf644fe1424688aed206142611df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:25:18.016325 7474 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45954:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 07:25:18.016472 7474 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 07:25:18.017176 8028 tablet_replica.cc:330] T 19a80042ed1b4a90af4eb83c2b20d4e1 P 6b563430b1a945acb4bf3ea743cbca17: stopping tablet replica
I20260925 07:25:18.017328 8028 raft_consensus.cc:2229] T 19a80042ed1b4a90af4eb83c2b20d4e1 P 6b563430b1a945acb4bf3ea743cbca17 [term 1 LEADER]: Raft consensus shutting down.
I20260925 07:25:18.017455 8028 raft_consensus.cc:2258] T 19a80042ed1b4a90af4eb83c2b20d4e1 P 6b563430b1a945acb4bf3ea743cbca17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:25:18.017658 7752 tablet_service.cc:1467] Processing CreateTablet for tablet b3415b91ded642a5bc776ad97610d38c (DEFAULT_TABLE table=test-table [id=06196f206a6746afb737ff377ca79b8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:25:18.017767 7752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3415b91ded642a5bc776ad97610d38c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:25:18.017833 8028 ts_tablet_manager.cc:1905] T 19a80042ed1b4a90af4eb83c2b20d4e1 P 6b563430b1a945acb4bf3ea743cbca17: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:25:18.018958 7892 tablet_service.cc:1467] Processing CreateTablet for tablet b3415b91ded642a5bc776ad97610d38c (DEFAULT_TABLE table=test-table [id=06196f206a6746afb737ff377ca79b8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:25:18.019044 7892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3415b91ded642a5bc776ad97610d38c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:25:18.019312 8028 ts_tablet_manager.cc:1918] T 19a80042ed1b4a90af4eb83c2b20d4e1 P 6b563430b1a945acb4bf3ea743cbca17: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 07:25:18.019385 8028 log.cc:1196] T 19a80042ed1b4a90af4eb83c2b20d4e1 P 6b563430b1a945acb4bf3ea743cbca17: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-1/wal/wals/19a80042ed1b4a90af4eb83c2b20d4e1
I20260925 07:25:18.019613 8028 ts_tablet_manager.cc:1939] T 19a80042ed1b4a90af4eb83c2b20d4e1 P 6b563430b1a945acb4bf3ea743cbca17: Deleting consensus metadata
I20260925 07:25:18.020191 7978 tablet_bootstrap.cc:492] T b3415b91ded642a5bc776ad97610d38c P a980bf644fe1424688aed206142611df: Bootstrap starting.
I20260925 07:25:18.020241 7982 tablet_bootstrap.cc:492] T b3415b91ded642a5bc776ad97610d38c P 6b563430b1a945acb4bf3ea743cbca17: Bootstrap starting.
I20260925 07:25:18.020581 7978 tablet_bootstrap.cc:654] T b3415b91ded642a5bc776ad97610d38c P a980bf644fe1424688aed206142611df: Neither blocks nor log segments found. Creating new log.
I20260925 07:25:18.020623 7982 tablet_bootstrap.cc:654] T b3415b91ded642a5bc776ad97610d38c P 6b563430b1a945acb4bf3ea743cbca17: Neither blocks nor log segments found. Creating new log.
I20260925 07:25:18.021214 7461 catalog_manager.cc:4841] TS 6b563430b1a945acb4bf3ea743cbca17 (127.5.158.66:35403): tablet 19a80042ed1b4a90af4eb83c2b20d4e1 (table test-table [id=9cd80b2460ae41388c80edae3c7da815]) successfully deleted
I20260925 07:25:18.021379 7612 tablet_service.cc:1467] Processing CreateTablet for tablet b3415b91ded642a5bc776ad97610d38c (DEFAULT_TABLE table=test-table [id=06196f206a6746afb737ff377ca79b8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:25:18.021483 7612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3415b91ded642a5bc776ad97610d38c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:25:18.022665 7979 tablet_bootstrap.cc:492] T b3415b91ded642a5bc776ad97610d38c P 9340b3d9a5314d24bdde11845526f19d: Bootstrap starting.
I20260925 07:25:18.022969 8027 ts_tablet_manager.cc:1905] T 19a80042ed1b4a90af4eb83c2b20d4e1 P a980bf644fe1424688aed206142611df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:25:18.023247 7979 tablet_bootstrap.cc:654] T b3415b91ded642a5bc776ad97610d38c P 9340b3d9a5314d24bdde11845526f19d: Neither blocks nor log segments found. Creating new log.
I20260925 07:25:18.024580 8027 ts_tablet_manager.cc:1918] T 19a80042ed1b4a90af4eb83c2b20d4e1 P a980bf644fe1424688aed206142611df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 07:25:18.024679 8027 log.cc:1196] T 19a80042ed1b4a90af4eb83c2b20d4e1 P a980bf644fe1424688aed206142611df: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-2/wal/wals/19a80042ed1b4a90af4eb83c2b20d4e1
I20260925 07:25:18.024731 7982 tablet_bootstrap.cc:492] T b3415b91ded642a5bc776ad97610d38c P 6b563430b1a945acb4bf3ea743cbca17: No bootstrap required, opened a new log
I20260925 07:25:18.024775 7982 ts_tablet_manager.cc:1397] T b3415b91ded642a5bc776ad97610d38c P 6b563430b1a945acb4bf3ea743cbca17: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260925 07:25:18.024885 7982 raft_consensus.cc:348] T b3415b91ded642a5bc776ad97610d38c P 6b563430b1a945acb4bf3ea743cbca17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } } peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } }
I20260925 07:25:18.024946 7982 raft_consensus.cc:374] T b3415b91ded642a5bc776ad97610d38c P 6b563430b1a945acb4bf3ea743cbca17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:25:18.024964 7982 raft_consensus.cc:729] T b3415b91ded642a5bc776ad97610d38c P 6b563430b1a945acb4bf3ea743cbca17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b563430b1a945acb4bf3ea743cbca17, State: Initialized, Role: FOLLOWER
I20260925 07:25:18.024959 8027 ts_tablet_manager.cc:1939] T 19a80042ed1b4a90af4eb83c2b20d4e1 P a980bf644fe1424688aed206142611df: Deleting consensus metadata
I20260925 07:25:18.025002 7982 consensus_queue.cc:260] T b3415b91ded642a5bc776ad97610d38c P 6b563430b1a945acb4bf3ea743cbca17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } } peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } }
I20260925 07:25:18.025302 7462 catalog_manager.cc:4841] TS a980bf644fe1424688aed206142611df (127.5.158.67:34023): tablet 19a80042ed1b4a90af4eb83c2b20d4e1 (table test-table [id=9cd80b2460ae41388c80edae3c7da815]) successfully deleted
I20260925 07:25:18.025609 7982 ts_tablet_manager.cc:1428] T b3415b91ded642a5bc776ad97610d38c P 6b563430b1a945acb4bf3ea743cbca17: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260925 07:25:18.025825 8029 tablet_replica.cc:330] T 19a80042ed1b4a90af4eb83c2b20d4e1 P 9340b3d9a5314d24bdde11845526f19d: stopping tablet replica
I20260925 07:25:18.025969 8029 raft_consensus.cc:2229] T 19a80042ed1b4a90af4eb83c2b20d4e1 P 9340b3d9a5314d24bdde11845526f19d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 07:25:18.026088 8029 raft_consensus.cc:2258] T 19a80042ed1b4a90af4eb83c2b20d4e1 P 9340b3d9a5314d24bdde11845526f19d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:25:18.026590 8029 ts_tablet_manager.cc:1905] T 19a80042ed1b4a90af4eb83c2b20d4e1 P 9340b3d9a5314d24bdde11845526f19d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:25:18.027983 8029 ts_tablet_manager.cc:1918] T 19a80042ed1b4a90af4eb83c2b20d4e1 P 9340b3d9a5314d24bdde11845526f19d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 07:25:18.028046 8029 log.cc:1196] T 19a80042ed1b4a90af4eb83c2b20d4e1 P 9340b3d9a5314d24bdde11845526f19d: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-0/wal/wals/19a80042ed1b4a90af4eb83c2b20d4e1
I20260925 07:25:18.028313 8029 ts_tablet_manager.cc:1939] T 19a80042ed1b4a90af4eb83c2b20d4e1 P 9340b3d9a5314d24bdde11845526f19d: Deleting consensus metadata
I20260925 07:25:18.028657 7461 catalog_manager.cc:4841] TS 9340b3d9a5314d24bdde11845526f19d (127.5.158.65:42225): tablet 19a80042ed1b4a90af4eb83c2b20d4e1 (table test-table [id=9cd80b2460ae41388c80edae3c7da815]) successfully deleted
I20260925 07:25:18.029052 7979 tablet_bootstrap.cc:492] T b3415b91ded642a5bc776ad97610d38c P 9340b3d9a5314d24bdde11845526f19d: No bootstrap required, opened a new log
I20260925 07:25:18.029117 7979 ts_tablet_manager.cc:1397] T b3415b91ded642a5bc776ad97610d38c P 9340b3d9a5314d24bdde11845526f19d: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260925 07:25:18.029392 7978 tablet_bootstrap.cc:492] T b3415b91ded642a5bc776ad97610d38c P a980bf644fe1424688aed206142611df: No bootstrap required, opened a new log
I20260925 07:25:18.029443 7978 ts_tablet_manager.cc:1397] T b3415b91ded642a5bc776ad97610d38c P a980bf644fe1424688aed206142611df: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20260925 07:25:18.029414 7979 raft_consensus.cc:348] T b3415b91ded642a5bc776ad97610d38c P 9340b3d9a5314d24bdde11845526f19d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } } peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } }
I20260925 07:25:18.029475 7979 raft_consensus.cc:374] T b3415b91ded642a5bc776ad97610d38c P 9340b3d9a5314d24bdde11845526f19d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:25:18.029505 7979 raft_consensus.cc:729] T b3415b91ded642a5bc776ad97610d38c P 9340b3d9a5314d24bdde11845526f19d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9340b3d9a5314d24bdde11845526f19d, State: Initialized, Role: FOLLOWER
I20260925 07:25:18.029546 7979 consensus_queue.cc:260] T b3415b91ded642a5bc776ad97610d38c P 9340b3d9a5314d24bdde11845526f19d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } } peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } }
I20260925 07:25:18.029564 7978 raft_consensus.cc:348] T b3415b91ded642a5bc776ad97610d38c P a980bf644fe1424688aed206142611df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } } peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } }
I20260925 07:25:18.029614 7978 raft_consensus.cc:374] T b3415b91ded642a5bc776ad97610d38c P a980bf644fe1424688aed206142611df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:25:18.029634 7978 raft_consensus.cc:729] T b3415b91ded642a5bc776ad97610d38c P a980bf644fe1424688aed206142611df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a980bf644fe1424688aed206142611df, State: Initialized, Role: FOLLOWER
I20260925 07:25:18.029628 7979 ts_tablet_manager.cc:1428] T b3415b91ded642a5bc776ad97610d38c P 9340b3d9a5314d24bdde11845526f19d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:18.029675 7978 consensus_queue.cc:260] T b3415b91ded642a5bc776ad97610d38c P a980bf644fe1424688aed206142611df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } } peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } }
I20260925 07:25:18.029754 7978 ts_tablet_manager.cc:1428] T b3415b91ded642a5bc776ad97610d38c P a980bf644fe1424688aed206142611df: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:18.272702 8001 raft_consensus.cc:482] T b3415b91ded642a5bc776ad97610d38c P 9340b3d9a5314d24bdde11845526f19d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 07:25:18.272791 8001 raft_consensus.cc:504] T b3415b91ded642a5bc776ad97610d38c P 9340b3d9a5314d24bdde11845526f19d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } } peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } }
I20260925 07:25:18.272976 8001 leader_election.cc:288] T b3415b91ded642a5bc776ad97610d38c P 9340b3d9a5314d24bdde11845526f19d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b563430b1a945acb4bf3ea743cbca17 (127.5.158.66:35403), a980bf644fe1424688aed206142611df (127.5.158.67:34023)
I20260925 07:25:18.273201 7771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3415b91ded642a5bc776ad97610d38c" candidate_uuid: "9340b3d9a5314d24bdde11845526f19d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b563430b1a945acb4bf3ea743cbca17" is_pre_election: true
I20260925 07:25:18.273279 7771 raft_consensus.cc:2454] T b3415b91ded642a5bc776ad97610d38c P 6b563430b1a945acb4bf3ea743cbca17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9340b3d9a5314d24bdde11845526f19d in term 0.
I20260925 07:25:18.273205 7912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3415b91ded642a5bc776ad97610d38c" candidate_uuid: "9340b3d9a5314d24bdde11845526f19d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a980bf644fe1424688aed206142611df" is_pre_election: true
I20260925 07:25:18.273315 7912 raft_consensus.cc:2454] T b3415b91ded642a5bc776ad97610d38c P a980bf644fe1424688aed206142611df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9340b3d9a5314d24bdde11845526f19d in term 0.
I20260925 07:25:18.273429 7565 leader_election.cc:302] T b3415b91ded642a5bc776ad97610d38c P 9340b3d9a5314d24bdde11845526f19d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b563430b1a945acb4bf3ea743cbca17, 9340b3d9a5314d24bdde11845526f19d; no voters:
I20260925 07:25:18.273530 8001 raft_consensus.cc:2781] T b3415b91ded642a5bc776ad97610d38c P 9340b3d9a5314d24bdde11845526f19d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 07:25:18.273564 8001 raft_consensus.cc:482] T b3415b91ded642a5bc776ad97610d38c P 9340b3d9a5314d24bdde11845526f19d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 07:25:18.273581 8001 raft_consensus.cc:3037] T b3415b91ded642a5bc776ad97610d38c P 9340b3d9a5314d24bdde11845526f19d [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:25:18.274240 8001 raft_consensus.cc:504] T b3415b91ded642a5bc776ad97610d38c P 9340b3d9a5314d24bdde11845526f19d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } } peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } }
I20260925 07:25:18.274360 8001 leader_election.cc:288] T b3415b91ded642a5bc776ad97610d38c P 9340b3d9a5314d24bdde11845526f19d [CANDIDATE]: Term 1 election: Requested vote from peers 6b563430b1a945acb4bf3ea743cbca17 (127.5.158.66:35403), a980bf644fe1424688aed206142611df (127.5.158.67:34023)
I20260925 07:25:18.274521 7771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3415b91ded642a5bc776ad97610d38c" candidate_uuid: "9340b3d9a5314d24bdde11845526f19d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b563430b1a945acb4bf3ea743cbca17"
I20260925 07:25:18.274513 7912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3415b91ded642a5bc776ad97610d38c" candidate_uuid: "9340b3d9a5314d24bdde11845526f19d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a980bf644fe1424688aed206142611df"
I20260925 07:25:18.274578 7912 raft_consensus.cc:3037] T b3415b91ded642a5bc776ad97610d38c P a980bf644fe1424688aed206142611df [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:25:18.274602 7771 raft_consensus.cc:3037] T b3415b91ded642a5bc776ad97610d38c P 6b563430b1a945acb4bf3ea743cbca17 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:25:18.275158 7771 raft_consensus.cc:2454] T b3415b91ded642a5bc776ad97610d38c P 6b563430b1a945acb4bf3ea743cbca17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9340b3d9a5314d24bdde11845526f19d in term 1.
I20260925 07:25:18.275162 7912 raft_consensus.cc:2454] T b3415b91ded642a5bc776ad97610d38c P a980bf644fe1424688aed206142611df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9340b3d9a5314d24bdde11845526f19d in term 1.
I20260925 07:25:18.275328 7565 leader_election.cc:302] T b3415b91ded642a5bc776ad97610d38c P 9340b3d9a5314d24bdde11845526f19d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b563430b1a945acb4bf3ea743cbca17, 9340b3d9a5314d24bdde11845526f19d; no voters:
I20260925 07:25:18.275432 8001 raft_consensus.cc:2781] T b3415b91ded642a5bc776ad97610d38c P 9340b3d9a5314d24bdde11845526f19d [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:25:18.275580 8001 raft_consensus.cc:686] T b3415b91ded642a5bc776ad97610d38c P 9340b3d9a5314d24bdde11845526f19d [term 1 LEADER]: Becoming Leader. State: Replica: 9340b3d9a5314d24bdde11845526f19d, State: Running, Role: LEADER
I20260925 07:25:18.275635 8001 consensus_queue.cc:237] T b3415b91ded642a5bc776ad97610d38c P 9340b3d9a5314d24bdde11845526f19d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } } peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } }
I20260925 07:25:18.276166 7474 catalog_manager.cc:5495] T b3415b91ded642a5bc776ad97610d38c P 9340b3d9a5314d24bdde11845526f19d reported cstate change: term changed from 0 to 1, leader changed from <none> to 9340b3d9a5314d24bdde11845526f19d (127.5.158.65). New cstate: current_term: 1 leader_uuid: "9340b3d9a5314d24bdde11845526f19d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } health_report { overall_health: UNKNOWN } } }
I20260925 07:25:18.307830 7977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:18.305610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46562 (local address 127.5.158.65:42225)
0925 07:25:18.305670 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:18.305672 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:18.305679 (+ 7us) server_negotiation.cc:407] Connection header received
0925 07:25:18.305731 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:18.305731 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:18.305742 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:18.305788 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 07:25:18.305900 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:18.306386 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:18.307110 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:18.307335 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:18.307413 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:18.307555 (+ 142us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 07:25:18.307585 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 07:25:18.307664 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:18.307731 (+ 67us) server_negotiation.cc:299] Negotiation successful
0925 07:25:18.307751 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 07:25:18.311211 7977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:18.309633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:33473 (local address 127.5.158.65:42225)
0925 07:25:18.309672 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:18.309674 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:18.309682 (+ 8us) server_negotiation.cc:407] Connection header received
0925 07:25:18.309698 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:18.309699 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:18.309705 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:18.309741 (+ 36us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 07:25:18.309829 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:18.310194 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:18.310776 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:18.310932 (+ 156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:18.310974 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:18.311069 (+ 95us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 07:25:18.311086 (+ 17us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 07:25:18.311115 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:18.311142 (+ 27us) server_negotiation.cc:299] Negotiation successful
0925 07:25:18.311152 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 07:25:18.311247 8023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:18.309564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:42225 (local address 127.5.158.65:33473)
0925 07:25:18.309611 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:18.309620 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:18.309655 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:18.309752 (+ 97us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:18.309753 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:18.309754 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 07:25:18.309789 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:18.309790 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:18.310242 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:18.310242 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:18.310716 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:18.310717 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:18.310986 (+ 269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:18.310987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:18.311022 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:18.311026 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 07:25:18.311114 (+ 88us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 07:25:18.311115 (+ 1us) client_negotiation.cc:769] Sending connection context
0925 07:25:18.311140 (+ 25us) client_negotiation.cc:240] Negotiation successful
0925 07:25:18.311166 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260925 07:25:18.313648 7912 raft_consensus.cc:1261] T b3415b91ded642a5bc776ad97610d38c P a980bf644fe1424688aed206142611df [term 1 FOLLOWER]: Refusing update from remote peer 9340b3d9a5314d24bdde11845526f19d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:25:18.313740 7771 raft_consensus.cc:1261] T b3415b91ded642a5bc776ad97610d38c P 6b563430b1a945acb4bf3ea743cbca17 [term 1 FOLLOWER]: Refusing update from remote peer 9340b3d9a5314d24bdde11845526f19d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:25:18.313819 7985 consensus_queue.cc:1035] T b3415b91ded642a5bc776ad97610d38c P 9340b3d9a5314d24bdde11845526f19d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:25:18.314052 7985 consensus_queue.cc:1035] T b3415b91ded642a5bc776ad97610d38c P 9340b3d9a5314d24bdde11845526f19d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 07:25:18.317703 8006 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260925 07:25:19.833869 7474 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45954:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 07:25:19.833949 7474 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45954:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 07:25:19.834612 7474 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 39542b3be6b7448ab00f36a31d0c7915: Sending DeleteTablet for 3 replicas of tablet b3415b91ded642a5bc776ad97610d38c
I20260925 07:25:19.834874 7892 tablet_service.cc:1514] Processing DeleteTablet for tablet b3415b91ded642a5bc776ad97610d38c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:25:19 UTC) from {username='kudu', principal='kudu/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:60872
I20260925 07:25:19.834879 7612 tablet_service.cc:1514] Processing DeleteTablet for tablet b3415b91ded642a5bc776ad97610d38c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:25:19 UTC) from {username='kudu', principal='kudu/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:46538
I20260925 07:25:19.834929 7752 tablet_service.cc:1514] Processing DeleteTablet for tablet b3415b91ded642a5bc776ad97610d38c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:25:19 UTC) from {username='kudu', principal='kudu/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:47972
I20260925 07:25:19.835098 8059 tablet_replica.cc:330] T b3415b91ded642a5bc776ad97610d38c P 6b563430b1a945acb4bf3ea743cbca17: stopping tablet replica
I20260925 07:25:19.835176 8059 raft_consensus.cc:2229] T b3415b91ded642a5bc776ad97610d38c P 6b563430b1a945acb4bf3ea743cbca17 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 07:25:19.835217 8059 raft_consensus.cc:2258] T b3415b91ded642a5bc776ad97610d38c P 6b563430b1a945acb4bf3ea743cbca17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:25:19.835431 8059 ts_tablet_manager.cc:1905] T b3415b91ded642a5bc776ad97610d38c P 6b563430b1a945acb4bf3ea743cbca17: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:25:19.837158 8057 tablet_replica.cc:330] T b3415b91ded642a5bc776ad97610d38c P 9340b3d9a5314d24bdde11845526f19d: stopping tablet replica
I20260925 07:25:19.837179 8059 ts_tablet_manager.cc:1918] T b3415b91ded642a5bc776ad97610d38c P 6b563430b1a945acb4bf3ea743cbca17: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 07:25:19.837231 8059 log.cc:1196] T b3415b91ded642a5bc776ad97610d38c P 6b563430b1a945acb4bf3ea743cbca17: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-1/wal/wals/b3415b91ded642a5bc776ad97610d38c
I20260925 07:25:19.837317 8057 raft_consensus.cc:2229] T b3415b91ded642a5bc776ad97610d38c P 9340b3d9a5314d24bdde11845526f19d [term 1 LEADER]: Raft consensus shutting down.
I20260925 07:25:19.837469 8057 raft_consensus.cc:2258] T b3415b91ded642a5bc776ad97610d38c P 9340b3d9a5314d24bdde11845526f19d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:25:19.837486 8059 ts_tablet_manager.cc:1939] T b3415b91ded642a5bc776ad97610d38c P 6b563430b1a945acb4bf3ea743cbca17: Deleting consensus metadata
I20260925 07:25:19.837608 8058 tablet_replica.cc:330] T b3415b91ded642a5bc776ad97610d38c P a980bf644fe1424688aed206142611df: stopping tablet replica
I20260925 07:25:19.837687 8058 raft_consensus.cc:2229] T b3415b91ded642a5bc776ad97610d38c P a980bf644fe1424688aed206142611df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 07:25:19.837714 8058 raft_consensus.cc:2258] T b3415b91ded642a5bc776ad97610d38c P a980bf644fe1424688aed206142611df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:25:19.837898 7461 catalog_manager.cc:4841] TS 6b563430b1a945acb4bf3ea743cbca17 (127.5.158.66:35403): tablet b3415b91ded642a5bc776ad97610d38c (table test-table [id=06196f206a6746afb737ff377ca79b8b]) successfully deleted
I20260925 07:25:19.837989 8057 ts_tablet_manager.cc:1905] T b3415b91ded642a5bc776ad97610d38c P 9340b3d9a5314d24bdde11845526f19d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:25:19.838017 8058 ts_tablet_manager.cc:1905] T b3415b91ded642a5bc776ad97610d38c P a980bf644fe1424688aed206142611df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:25:19.839118 8057 ts_tablet_manager.cc:1918] T b3415b91ded642a5bc776ad97610d38c P 9340b3d9a5314d24bdde11845526f19d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 07:25:19.839200 8057 log.cc:1196] T b3415b91ded642a5bc776ad97610d38c P 9340b3d9a5314d24bdde11845526f19d: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-0/wal/wals/b3415b91ded642a5bc776ad97610d38c
I20260925 07:25:19.839282 8058 ts_tablet_manager.cc:1918] T b3415b91ded642a5bc776ad97610d38c P a980bf644fe1424688aed206142611df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 07:25:19.839344 8058 log.cc:1196] T b3415b91ded642a5bc776ad97610d38c P a980bf644fe1424688aed206142611df: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-2/wal/wals/b3415b91ded642a5bc776ad97610d38c
I20260925 07:25:19.839519 8057 ts_tablet_manager.cc:1939] T b3415b91ded642a5bc776ad97610d38c P 9340b3d9a5314d24bdde11845526f19d: Deleting consensus metadata
I20260925 07:25:19.839593 8058 ts_tablet_manager.cc:1939] T b3415b91ded642a5bc776ad97610d38c P a980bf644fe1424688aed206142611df: Deleting consensus metadata
I20260925 07:25:19.839797 7461 catalog_manager.cc:4841] TS 9340b3d9a5314d24bdde11845526f19d (127.5.158.65:42225): tablet b3415b91ded642a5bc776ad97610d38c (table test-table [id=06196f206a6746afb737ff377ca79b8b]) successfully deleted
I20260925 07:25:19.839836 7462 catalog_manager.cc:4841] TS a980bf644fe1424688aed206142611df (127.5.158.67:34023): tablet b3415b91ded642a5bc776ad97610d38c (table test-table [id=06196f206a6746afb737ff377ca79b8b]) successfully deleted
Sep 25 07:25:19 dist-test-slave-1drq krb5kdc[7430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321119, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 07:25:19 dist-test-slave-1drq krb5kdc[7430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321119, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:25:19.863394 8067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:19.851423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45960 (local address 127.5.158.126:37263)
0925 07:25:19.851545 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:19.851546 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:19.851567 (+ 21us) server_negotiation.cc:407] Connection header received
0925 07:25:19.851624 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:19.851625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:19.851634 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:19.851688 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:19.852118 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:19.852551 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:19.853181 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:19.853346 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:19.853387 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:19.861603 (+ 8216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:19.861604 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:19.861604 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:19.861610 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:19.862662 (+ 1052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:19.862915 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:19.862916 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:19.862916 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:19.862960 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:19.863080 (+ 120us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:19.863080 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:19.863081 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:19.863157 (+ 76us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:19.863217 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:19.863287 (+ 70us) server_negotiation.cc:299] Negotiation successful
0925 07:25:19.863339 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":78,"threads_started":1}
I20260925 07:25:19.864554 7474 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45960:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 07:25:19.864711 7474 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 07:25:19.865800 7612 tablet_service.cc:1467] Processing CreateTablet for tablet 428fdac85c3148e880931edcbb222ba7 (DEFAULT_TABLE table=test-table [id=851da8415a0d44d6a5dd08b83fdc6484]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:25:19.865830 7892 tablet_service.cc:1467] Processing CreateTablet for tablet 428fdac85c3148e880931edcbb222ba7 (DEFAULT_TABLE table=test-table [id=851da8415a0d44d6a5dd08b83fdc6484]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:25:19.865830 7752 tablet_service.cc:1467] Processing CreateTablet for tablet 428fdac85c3148e880931edcbb222ba7 (DEFAULT_TABLE table=test-table [id=851da8415a0d44d6a5dd08b83fdc6484]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:25:19.865917 7612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 428fdac85c3148e880931edcbb222ba7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:25:19.865923 7752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 428fdac85c3148e880931edcbb222ba7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:25:19.865921 7892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 428fdac85c3148e880931edcbb222ba7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:25:19.866830 8068 tablet_bootstrap.cc:492] T 428fdac85c3148e880931edcbb222ba7 P 9340b3d9a5314d24bdde11845526f19d: Bootstrap starting.
I20260925 07:25:19.867149 8070 tablet_bootstrap.cc:492] T 428fdac85c3148e880931edcbb222ba7 P a980bf644fe1424688aed206142611df: Bootstrap starting.
I20260925 07:25:19.867182 8068 tablet_bootstrap.cc:654] T 428fdac85c3148e880931edcbb222ba7 P 9340b3d9a5314d24bdde11845526f19d: Neither blocks nor log segments found. Creating new log.
I20260925 07:25:19.867592 8070 tablet_bootstrap.cc:654] T 428fdac85c3148e880931edcbb222ba7 P a980bf644fe1424688aed206142611df: Neither blocks nor log segments found. Creating new log.
I20260925 07:25:19.867735 8068 tablet_bootstrap.cc:492] T 428fdac85c3148e880931edcbb222ba7 P 9340b3d9a5314d24bdde11845526f19d: No bootstrap required, opened a new log
I20260925 07:25:19.867784 8068 ts_tablet_manager.cc:1397] T 428fdac85c3148e880931edcbb222ba7 P 9340b3d9a5314d24bdde11845526f19d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 07:25:19.867882 8069 tablet_bootstrap.cc:492] T 428fdac85c3148e880931edcbb222ba7 P 6b563430b1a945acb4bf3ea743cbca17: Bootstrap starting.
I20260925 07:25:19.867889 8068 raft_consensus.cc:348] T 428fdac85c3148e880931edcbb222ba7 P 9340b3d9a5314d24bdde11845526f19d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } } peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } }
I20260925 07:25:19.867945 8068 raft_consensus.cc:374] T 428fdac85c3148e880931edcbb222ba7 P 9340b3d9a5314d24bdde11845526f19d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:25:19.867969 8068 raft_consensus.cc:729] T 428fdac85c3148e880931edcbb222ba7 P 9340b3d9a5314d24bdde11845526f19d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9340b3d9a5314d24bdde11845526f19d, State: Initialized, Role: FOLLOWER
I20260925 07:25:19.868036 8068 consensus_queue.cc:260] T 428fdac85c3148e880931edcbb222ba7 P 9340b3d9a5314d24bdde11845526f19d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } } peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } }
I20260925 07:25:19.868126 8068 ts_tablet_manager.cc:1428] T 428fdac85c3148e880931edcbb222ba7 P 9340b3d9a5314d24bdde11845526f19d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:19.868183 8069 tablet_bootstrap.cc:654] T 428fdac85c3148e880931edcbb222ba7 P 6b563430b1a945acb4bf3ea743cbca17: Neither blocks nor log segments found. Creating new log.
I20260925 07:25:19.868485 8070 tablet_bootstrap.cc:492] T 428fdac85c3148e880931edcbb222ba7 P a980bf644fe1424688aed206142611df: No bootstrap required, opened a new log
I20260925 07:25:19.868530 8070 ts_tablet_manager.cc:1397] T 428fdac85c3148e880931edcbb222ba7 P a980bf644fe1424688aed206142611df: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 07:25:19.868673 8070 raft_consensus.cc:348] T 428fdac85c3148e880931edcbb222ba7 P a980bf644fe1424688aed206142611df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } } peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } }
I20260925 07:25:19.868741 8070 raft_consensus.cc:374] T 428fdac85c3148e880931edcbb222ba7 P a980bf644fe1424688aed206142611df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:25:19.868763 8070 raft_consensus.cc:729] T 428fdac85c3148e880931edcbb222ba7 P a980bf644fe1424688aed206142611df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a980bf644fe1424688aed206142611df, State: Initialized, Role: FOLLOWER
I20260925 07:25:19.868817 8070 consensus_queue.cc:260] T 428fdac85c3148e880931edcbb222ba7 P a980bf644fe1424688aed206142611df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } } peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } }
I20260925 07:25:19.868849 8069 tablet_bootstrap.cc:492] T 428fdac85c3148e880931edcbb222ba7 P 6b563430b1a945acb4bf3ea743cbca17: No bootstrap required, opened a new log
I20260925 07:25:19.868897 8069 ts_tablet_manager.cc:1397] T 428fdac85c3148e880931edcbb222ba7 P 6b563430b1a945acb4bf3ea743cbca17: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 07:25:19.868963 8070 ts_tablet_manager.cc:1428] T 428fdac85c3148e880931edcbb222ba7 P a980bf644fe1424688aed206142611df: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:19.869029 8069 raft_consensus.cc:348] T 428fdac85c3148e880931edcbb222ba7 P 6b563430b1a945acb4bf3ea743cbca17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } } peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } }
I20260925 07:25:19.869179 8069 raft_consensus.cc:374] T 428fdac85c3148e880931edcbb222ba7 P 6b563430b1a945acb4bf3ea743cbca17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:25:19.869223 8069 raft_consensus.cc:729] T 428fdac85c3148e880931edcbb222ba7 P 6b563430b1a945acb4bf3ea743cbca17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b563430b1a945acb4bf3ea743cbca17, State: Initialized, Role: FOLLOWER
I20260925 07:25:19.869306 8069 consensus_queue.cc:260] T 428fdac85c3148e880931edcbb222ba7 P 6b563430b1a945acb4bf3ea743cbca17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } } peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } }
I20260925 07:25:19.869480 8069 ts_tablet_manager.cc:1428] T 428fdac85c3148e880931edcbb222ba7 P 6b563430b1a945acb4bf3ea743cbca17: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260925 07:25:20.039629 8040 raft_consensus.cc:482] T 428fdac85c3148e880931edcbb222ba7 P 9340b3d9a5314d24bdde11845526f19d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 07:25:20.039713 8040 raft_consensus.cc:504] T 428fdac85c3148e880931edcbb222ba7 P 9340b3d9a5314d24bdde11845526f19d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } } peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } }
I20260925 07:25:20.039897 8040 leader_election.cc:288] T 428fdac85c3148e880931edcbb222ba7 P 9340b3d9a5314d24bdde11845526f19d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b563430b1a945acb4bf3ea743cbca17 (127.5.158.66:35403), a980bf644fe1424688aed206142611df (127.5.158.67:34023)
I20260925 07:25:20.040108 7771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "428fdac85c3148e880931edcbb222ba7" candidate_uuid: "9340b3d9a5314d24bdde11845526f19d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b563430b1a945acb4bf3ea743cbca17" is_pre_election: true
I20260925 07:25:20.040143 7911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "428fdac85c3148e880931edcbb222ba7" candidate_uuid: "9340b3d9a5314d24bdde11845526f19d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a980bf644fe1424688aed206142611df" is_pre_election: true
I20260925 07:25:20.040205 7771 raft_consensus.cc:2454] T 428fdac85c3148e880931edcbb222ba7 P 6b563430b1a945acb4bf3ea743cbca17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9340b3d9a5314d24bdde11845526f19d in term 0.
I20260925 07:25:20.040251 7911 raft_consensus.cc:2454] T 428fdac85c3148e880931edcbb222ba7 P a980bf644fe1424688aed206142611df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9340b3d9a5314d24bdde11845526f19d in term 0.
I20260925 07:25:20.040349 7565 leader_election.cc:302] T 428fdac85c3148e880931edcbb222ba7 P 9340b3d9a5314d24bdde11845526f19d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b563430b1a945acb4bf3ea743cbca17, 9340b3d9a5314d24bdde11845526f19d; no voters:
I20260925 07:25:20.040470 8040 raft_consensus.cc:2781] T 428fdac85c3148e880931edcbb222ba7 P 9340b3d9a5314d24bdde11845526f19d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 07:25:20.040519 8040 raft_consensus.cc:482] T 428fdac85c3148e880931edcbb222ba7 P 9340b3d9a5314d24bdde11845526f19d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 07:25:20.040535 8040 raft_consensus.cc:3037] T 428fdac85c3148e880931edcbb222ba7 P 9340b3d9a5314d24bdde11845526f19d [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:25:20.041280 8040 raft_consensus.cc:504] T 428fdac85c3148e880931edcbb222ba7 P 9340b3d9a5314d24bdde11845526f19d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } } peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } }
I20260925 07:25:20.041401 8040 leader_election.cc:288] T 428fdac85c3148e880931edcbb222ba7 P 9340b3d9a5314d24bdde11845526f19d [CANDIDATE]: Term 1 election: Requested vote from peers 6b563430b1a945acb4bf3ea743cbca17 (127.5.158.66:35403), a980bf644fe1424688aed206142611df (127.5.158.67:34023)
I20260925 07:25:20.041582 7911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "428fdac85c3148e880931edcbb222ba7" candidate_uuid: "9340b3d9a5314d24bdde11845526f19d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a980bf644fe1424688aed206142611df"
I20260925 07:25:20.041678 7911 raft_consensus.cc:3037] T 428fdac85c3148e880931edcbb222ba7 P a980bf644fe1424688aed206142611df [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:25:20.041697 7771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "428fdac85c3148e880931edcbb222ba7" candidate_uuid: "9340b3d9a5314d24bdde11845526f19d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b563430b1a945acb4bf3ea743cbca17"
I20260925 07:25:20.041770 7771 raft_consensus.cc:3037] T 428fdac85c3148e880931edcbb222ba7 P 6b563430b1a945acb4bf3ea743cbca17 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:25:20.042280 7911 raft_consensus.cc:2454] T 428fdac85c3148e880931edcbb222ba7 P a980bf644fe1424688aed206142611df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9340b3d9a5314d24bdde11845526f19d in term 1.
I20260925 07:25:20.042409 7771 raft_consensus.cc:2454] T 428fdac85c3148e880931edcbb222ba7 P 6b563430b1a945acb4bf3ea743cbca17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9340b3d9a5314d24bdde11845526f19d in term 1.
I20260925 07:25:20.042465 7566 leader_election.cc:302] T 428fdac85c3148e880931edcbb222ba7 P 9340b3d9a5314d24bdde11845526f19d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9340b3d9a5314d24bdde11845526f19d, a980bf644fe1424688aed206142611df; no voters:
I20260925 07:25:20.042546 8040 raft_consensus.cc:2781] T 428fdac85c3148e880931edcbb222ba7 P 9340b3d9a5314d24bdde11845526f19d [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:25:20.042585 8040 raft_consensus.cc:686] T 428fdac85c3148e880931edcbb222ba7 P 9340b3d9a5314d24bdde11845526f19d [term 1 LEADER]: Becoming Leader. State: Replica: 9340b3d9a5314d24bdde11845526f19d, State: Running, Role: LEADER
I20260925 07:25:20.042645 8040 consensus_queue.cc:237] T 428fdac85c3148e880931edcbb222ba7 P 9340b3d9a5314d24bdde11845526f19d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } } peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } }
I20260925 07:25:20.043183 7474 catalog_manager.cc:5495] T 428fdac85c3148e880931edcbb222ba7 P 9340b3d9a5314d24bdde11845526f19d reported cstate change: term changed from 0 to 1, leader changed from <none> to 9340b3d9a5314d24bdde11845526f19d (127.5.158.65). New cstate: current_term: 1 leader_uuid: "9340b3d9a5314d24bdde11845526f19d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } health_report { overall_health: UNKNOWN } } }
I20260925 07:25:20.050575 8078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:20.048375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46568 (local address 127.5.158.65:42225)
0925 07:25:20.048513 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:20.048515 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:20.048521 (+ 6us) server_negotiation.cc:407] Connection header received
0925 07:25:20.048539 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:20.048539 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:20.048548 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:20.048606 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 07:25:20.048731 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:20.049231 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:20.049911 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:20.050098 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:20.050157 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:20.050335 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 07:25:20.050363 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 07:25:20.050413 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:20.050505 (+ 92us) server_negotiation.cc:299] Negotiation successful
0925 07:25:20.050529 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":79,"threads_started":1}
I20260925 07:25:20.052305 7911 raft_consensus.cc:1261] T 428fdac85c3148e880931edcbb222ba7 P a980bf644fe1424688aed206142611df [term 1 FOLLOWER]: Refusing update from remote peer 9340b3d9a5314d24bdde11845526f19d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:25:20.052309 7771 raft_consensus.cc:1261] T 428fdac85c3148e880931edcbb222ba7 P 6b563430b1a945acb4bf3ea743cbca17 [term 1 FOLLOWER]: Refusing update from remote peer 9340b3d9a5314d24bdde11845526f19d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:25:20.052481 7985 consensus_queue.cc:1035] T 428fdac85c3148e880931edcbb222ba7 P 9340b3d9a5314d24bdde11845526f19d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:25:20.052594 8040 consensus_queue.cc:1035] T 428fdac85c3148e880931edcbb222ba7 P 9340b3d9a5314d24bdde11845526f19d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:25:20.061161 7473 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45960:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 07:25:20.061234 7473 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45960:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 07:25:20.061748 7473 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 39542b3be6b7448ab00f36a31d0c7915: Sending DeleteTablet for 3 replicas of tablet 428fdac85c3148e880931edcbb222ba7
I20260925 07:25:20.061973 7612 tablet_service.cc:1514] Processing DeleteTablet for tablet 428fdac85c3148e880931edcbb222ba7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:25:20 UTC) from {username='kudu', principal='kudu/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:46538
I20260925 07:25:20.062026 7892 tablet_service.cc:1514] Processing DeleteTablet for tablet 428fdac85c3148e880931edcbb222ba7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:25:20 UTC) from {username='kudu', principal='kudu/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:60872
I20260925 07:25:20.062110 7752 tablet_service.cc:1514] Processing DeleteTablet for tablet 428fdac85c3148e880931edcbb222ba7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:25:20 UTC) from {username='kudu', principal='kudu/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:47972
I20260925 07:25:20.062112 8058 tablet_replica.cc:330] T 428fdac85c3148e880931edcbb222ba7 P a980bf644fe1424688aed206142611df: stopping tablet replica
I20260925 07:25:20.062171 8058 raft_consensus.cc:2229] T 428fdac85c3148e880931edcbb222ba7 P a980bf644fe1424688aed206142611df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 07:25:20.062189 8057 tablet_replica.cc:330] T 428fdac85c3148e880931edcbb222ba7 P 9340b3d9a5314d24bdde11845526f19d: stopping tablet replica
I20260925 07:25:20.062212 8058 raft_consensus.cc:2258] T 428fdac85c3148e880931edcbb222ba7 P a980bf644fe1424688aed206142611df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:25:20.062230 8057 raft_consensus.cc:2229] T 428fdac85c3148e880931edcbb222ba7 P 9340b3d9a5314d24bdde11845526f19d [term 1 LEADER]: Raft consensus shutting down.
I20260925 07:25:20.062176 7473 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45960:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 07:25:20.062268 8057 raft_consensus.cc:2258] T 428fdac85c3148e880931edcbb222ba7 P 9340b3d9a5314d24bdde11845526f19d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260925 07:25:20.062304 7473 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 07:25:20.062325 8059 tablet_replica.cc:330] T 428fdac85c3148e880931edcbb222ba7 P 6b563430b1a945acb4bf3ea743cbca17: stopping tablet replica
I20260925 07:25:20.062368 8059 raft_consensus.cc:2229] T 428fdac85c3148e880931edcbb222ba7 P 6b563430b1a945acb4bf3ea743cbca17 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 07:25:20.062426 8059 raft_consensus.cc:2258] T 428fdac85c3148e880931edcbb222ba7 P 6b563430b1a945acb4bf3ea743cbca17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:25:20.062701 8058 ts_tablet_manager.cc:1905] T 428fdac85c3148e880931edcbb222ba7 P a980bf644fe1424688aed206142611df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:25:20.062728 8057 ts_tablet_manager.cc:1905] T 428fdac85c3148e880931edcbb222ba7 P 9340b3d9a5314d24bdde11845526f19d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:25:20.062788 8059 ts_tablet_manager.cc:1905] T 428fdac85c3148e880931edcbb222ba7 P 6b563430b1a945acb4bf3ea743cbca17: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:25:20.063915 8057 ts_tablet_manager.cc:1918] T 428fdac85c3148e880931edcbb222ba7 P 9340b3d9a5314d24bdde11845526f19d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 07:25:20.063983 8057 log.cc:1196] T 428fdac85c3148e880931edcbb222ba7 P 9340b3d9a5314d24bdde11845526f19d: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-0/wal/wals/428fdac85c3148e880931edcbb222ba7
I20260925 07:25:20.064088 8058 ts_tablet_manager.cc:1918] T 428fdac85c3148e880931edcbb222ba7 P a980bf644fe1424688aed206142611df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 07:25:20.064137 8058 log.cc:1196] T 428fdac85c3148e880931edcbb222ba7 P a980bf644fe1424688aed206142611df: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-2/wal/wals/428fdac85c3148e880931edcbb222ba7
I20260925 07:25:20.064152 8059 ts_tablet_manager.cc:1918] T 428fdac85c3148e880931edcbb222ba7 P 6b563430b1a945acb4bf3ea743cbca17: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 07:25:20.064178 8057 ts_tablet_manager.cc:1939] T 428fdac85c3148e880931edcbb222ba7 P 9340b3d9a5314d24bdde11845526f19d: Deleting consensus metadata
I20260925 07:25:20.064193 8059 log.cc:1196] T 428fdac85c3148e880931edcbb222ba7 P 6b563430b1a945acb4bf3ea743cbca17: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-1/wal/wals/428fdac85c3148e880931edcbb222ba7
I20260925 07:25:20.064345 8058 ts_tablet_manager.cc:1939] T 428fdac85c3148e880931edcbb222ba7 P a980bf644fe1424688aed206142611df: Deleting consensus metadata
I20260925 07:25:20.064407 8059 ts_tablet_manager.cc:1939] T 428fdac85c3148e880931edcbb222ba7 P 6b563430b1a945acb4bf3ea743cbca17: Deleting consensus metadata
I20260925 07:25:20.064435 7461 catalog_manager.cc:4841] TS 9340b3d9a5314d24bdde11845526f19d (127.5.158.65:42225): tablet 428fdac85c3148e880931edcbb222ba7 (table test-table [id=851da8415a0d44d6a5dd08b83fdc6484]) successfully deleted
I20260925 07:25:20.064581 7462 catalog_manager.cc:4841] TS a980bf644fe1424688aed206142611df (127.5.158.67:34023): tablet 428fdac85c3148e880931edcbb222ba7 (table test-table [id=851da8415a0d44d6a5dd08b83fdc6484]) successfully deleted
I20260925 07:25:20.064677 7461 catalog_manager.cc:4841] TS 6b563430b1a945acb4bf3ea743cbca17 (127.5.158.66:35403): tablet 428fdac85c3148e880931edcbb222ba7 (table test-table [id=851da8415a0d44d6a5dd08b83fdc6484]) successfully deleted
I20260925 07:25:20.065441 7612 tablet_service.cc:1467] Processing CreateTablet for tablet 5ae0fc3a2a5b4cdd8ee0a469b9a72144 (DEFAULT_TABLE table=test-table [id=043d8cd6ee264f2a887f0a37121e3bce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:25:20.065448 7752 tablet_service.cc:1467] Processing CreateTablet for tablet 5ae0fc3a2a5b4cdd8ee0a469b9a72144 (DEFAULT_TABLE table=test-table [id=043d8cd6ee264f2a887f0a37121e3bce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:25:20.065549 7752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ae0fc3a2a5b4cdd8ee0a469b9a72144. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:25:20.065557 7612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ae0fc3a2a5b4cdd8ee0a469b9a72144. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:25:20.066300 8069 tablet_bootstrap.cc:492] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P 6b563430b1a945acb4bf3ea743cbca17: Bootstrap starting.
I20260925 07:25:20.066350 8068 tablet_bootstrap.cc:492] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P 9340b3d9a5314d24bdde11845526f19d: Bootstrap starting.
I20260925 07:25:20.066355 7892 tablet_service.cc:1467] Processing CreateTablet for tablet 5ae0fc3a2a5b4cdd8ee0a469b9a72144 (DEFAULT_TABLE table=test-table [id=043d8cd6ee264f2a887f0a37121e3bce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:25:20.066452 7892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ae0fc3a2a5b4cdd8ee0a469b9a72144. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:25:20.066648 8069 tablet_bootstrap.cc:654] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P 6b563430b1a945acb4bf3ea743cbca17: Neither blocks nor log segments found. Creating new log.
I20260925 07:25:20.066680 8068 tablet_bootstrap.cc:654] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P 9340b3d9a5314d24bdde11845526f19d: Neither blocks nor log segments found. Creating new log.
I20260925 07:25:20.067189 8069 tablet_bootstrap.cc:492] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P 6b563430b1a945acb4bf3ea743cbca17: No bootstrap required, opened a new log
I20260925 07:25:20.067224 8068 tablet_bootstrap.cc:492] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P 9340b3d9a5314d24bdde11845526f19d: No bootstrap required, opened a new log
I20260925 07:25:20.067233 8069 ts_tablet_manager.cc:1397] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P 6b563430b1a945acb4bf3ea743cbca17: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 07:25:20.067263 8068 ts_tablet_manager.cc:1397] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P 9340b3d9a5314d24bdde11845526f19d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 07:25:20.067337 8069 raft_consensus.cc:348] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P 6b563430b1a945acb4bf3ea743cbca17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } } peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } }
I20260925 07:25:20.067399 8069 raft_consensus.cc:374] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P 6b563430b1a945acb4bf3ea743cbca17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:25:20.067376 8068 raft_consensus.cc:348] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P 9340b3d9a5314d24bdde11845526f19d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } } peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } }
I20260925 07:25:20.067417 8069 raft_consensus.cc:729] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P 6b563430b1a945acb4bf3ea743cbca17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b563430b1a945acb4bf3ea743cbca17, State: Initialized, Role: FOLLOWER
I20260925 07:25:20.067437 8068 raft_consensus.cc:374] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P 9340b3d9a5314d24bdde11845526f19d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:25:20.067472 8068 raft_consensus.cc:729] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P 9340b3d9a5314d24bdde11845526f19d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9340b3d9a5314d24bdde11845526f19d, State: Initialized, Role: FOLLOWER
I20260925 07:25:20.067482 8069 consensus_queue.cc:260] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P 6b563430b1a945acb4bf3ea743cbca17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } } peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } }
I20260925 07:25:20.067509 8068 consensus_queue.cc:260] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P 9340b3d9a5314d24bdde11845526f19d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } } peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } }
I20260925 07:25:20.067589 8069 ts_tablet_manager.cc:1428] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P 6b563430b1a945acb4bf3ea743cbca17: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:20.067595 8068 ts_tablet_manager.cc:1428] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P 9340b3d9a5314d24bdde11845526f19d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:20.067584 8070 tablet_bootstrap.cc:492] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P a980bf644fe1424688aed206142611df: Bootstrap starting.
I20260925 07:25:20.067996 8070 tablet_bootstrap.cc:654] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P a980bf644fe1424688aed206142611df: Neither blocks nor log segments found. Creating new log.
I20260925 07:25:20.068655 8070 tablet_bootstrap.cc:492] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P a980bf644fe1424688aed206142611df: No bootstrap required, opened a new log
I20260925 07:25:20.068692 8070 ts_tablet_manager.cc:1397] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P a980bf644fe1424688aed206142611df: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 07:25:20.068837 8070 raft_consensus.cc:348] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P a980bf644fe1424688aed206142611df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } } peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } }
I20260925 07:25:20.068912 8070 raft_consensus.cc:374] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P a980bf644fe1424688aed206142611df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:25:20.068933 8070 raft_consensus.cc:729] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P a980bf644fe1424688aed206142611df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a980bf644fe1424688aed206142611df, State: Initialized, Role: FOLLOWER
I20260925 07:25:20.068993 8070 consensus_queue.cc:260] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P a980bf644fe1424688aed206142611df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } } peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } }
I20260925 07:25:20.069068 8070 ts_tablet_manager.cc:1428] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P a980bf644fe1424688aed206142611df: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:20.274281 8074 raft_consensus.cc:482] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P a980bf644fe1424688aed206142611df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 07:25:20.274349 8074 raft_consensus.cc:504] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P a980bf644fe1424688aed206142611df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } } peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } }
I20260925 07:25:20.274653 8074 leader_election.cc:288] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P a980bf644fe1424688aed206142611df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9340b3d9a5314d24bdde11845526f19d (127.5.158.65:42225), 6b563430b1a945acb4bf3ea743cbca17 (127.5.158.66:35403)
I20260925 07:25:20.277635 8078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:20.274908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:59855 (local address 127.5.158.65:42225)
0925 07:25:20.274962 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:20.274963 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:20.274973 (+ 10us) server_negotiation.cc:407] Connection header received
0925 07:25:20.275094 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:20.275095 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:20.275108 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:20.275137 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:25:20.275357 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:20.275835 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:20.276856 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:20.277282 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:20.277338 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:20.277362 (+ 24us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:20.277521 (+ 159us) server_negotiation.cc:299] Negotiation successful
0925 07:25:20.277574 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260925 07:25:20.277653 8085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:20.274808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:42225 (local address 127.5.158.67:59855)
0925 07:25:20.274919 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:20.274929 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:20.274975 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:20.275216 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:20.275217 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:20.275219 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:25:20.275285 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:20.275287 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:20.275909 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:20.275909 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:20.276754 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:20.276757 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:20.277357 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:20.277358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:20.277430 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:20.277434 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 07:25:20.277504 (+ 70us) client_negotiation.cc:240] Negotiation successful
0925 07:25:20.277546 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78,"mutex_wait_us":40,"thread_start_us":57,"threads_started":1}
I20260925 07:25:20.277658 8086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:20.274967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.66:35403 (local address 127.5.158.67:48181)
0925 07:25:20.275085 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:20.275094 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:20.275145 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:20.275262 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:20.275262 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:20.275264 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:25:20.275332 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:20.275334 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:20.275951 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:20.275951 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:20.276756 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:20.276758 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:20.277387 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:20.277392 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:20.277449 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:20.277452 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 07:25:20.277533 (+ 81us) client_negotiation.cc:240] Negotiation successful
0925 07:25:20.277580 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"mutex_wait_us":22,"thread_start_us":65,"threads_started":1}
I20260925 07:25:20.277711 8087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:20.275017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:48181 (local address 127.5.158.66:35403)
0925 07:25:20.275140 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:20.275142 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:20.275150 (+ 8us) server_negotiation.cc:407] Connection header received
0925 07:25:20.275184 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:20.275185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:20.275194 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:20.275255 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:25:20.275395 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:20.275848 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:20.276848 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:20.277290 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:20.277378 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:20.277386 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:20.277585 (+ 199us) server_negotiation.cc:299] Negotiation successful
0925 07:25:20.277633 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":61,"threads_started":1}
I20260925 07:25:20.277884 7771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae0fc3a2a5b4cdd8ee0a469b9a72144" candidate_uuid: "a980bf644fe1424688aed206142611df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b563430b1a945acb4bf3ea743cbca17" is_pre_election: true
I20260925 07:25:20.277920 7632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae0fc3a2a5b4cdd8ee0a469b9a72144" candidate_uuid: "a980bf644fe1424688aed206142611df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9340b3d9a5314d24bdde11845526f19d" is_pre_election: true
I20260925 07:25:20.277995 7771 raft_consensus.cc:2454] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P 6b563430b1a945acb4bf3ea743cbca17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a980bf644fe1424688aed206142611df in term 0.
I20260925 07:25:20.278003 7632 raft_consensus.cc:2454] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P 9340b3d9a5314d24bdde11845526f19d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a980bf644fe1424688aed206142611df in term 0.
I20260925 07:25:20.278152 7845 leader_election.cc:302] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P a980bf644fe1424688aed206142611df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9340b3d9a5314d24bdde11845526f19d, a980bf644fe1424688aed206142611df; no voters:
I20260925 07:25:20.278285 8074 raft_consensus.cc:2781] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P a980bf644fe1424688aed206142611df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 07:25:20.278330 8074 raft_consensus.cc:482] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P a980bf644fe1424688aed206142611df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 07:25:20.278343 8074 raft_consensus.cc:3037] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P a980bf644fe1424688aed206142611df [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:25:20.278925 8074 raft_consensus.cc:504] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P a980bf644fe1424688aed206142611df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } } peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } }
I20260925 07:25:20.279043 8074 leader_election.cc:288] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P a980bf644fe1424688aed206142611df [CANDIDATE]: Term 1 election: Requested vote from peers 9340b3d9a5314d24bdde11845526f19d (127.5.158.65:42225), 6b563430b1a945acb4bf3ea743cbca17 (127.5.158.66:35403)
I20260925 07:25:20.279186 7632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae0fc3a2a5b4cdd8ee0a469b9a72144" candidate_uuid: "a980bf644fe1424688aed206142611df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9340b3d9a5314d24bdde11845526f19d"
I20260925 07:25:20.279215 7771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae0fc3a2a5b4cdd8ee0a469b9a72144" candidate_uuid: "a980bf644fe1424688aed206142611df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b563430b1a945acb4bf3ea743cbca17"
I20260925 07:25:20.279253 7632 raft_consensus.cc:3037] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P 9340b3d9a5314d24bdde11845526f19d [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:25:20.279289 7771 raft_consensus.cc:3037] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P 6b563430b1a945acb4bf3ea743cbca17 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:25:20.279844 7771 raft_consensus.cc:2454] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P 6b563430b1a945acb4bf3ea743cbca17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a980bf644fe1424688aed206142611df in term 1.
I20260925 07:25:20.279860 7632 raft_consensus.cc:2454] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P 9340b3d9a5314d24bdde11845526f19d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a980bf644fe1424688aed206142611df in term 1.
I20260925 07:25:20.280012 7845 leader_election.cc:302] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P a980bf644fe1424688aed206142611df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b563430b1a945acb4bf3ea743cbca17, a980bf644fe1424688aed206142611df; no voters:
I20260925 07:25:20.280110 8074 raft_consensus.cc:2781] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P a980bf644fe1424688aed206142611df [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:25:20.280229 8074 raft_consensus.cc:686] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P a980bf644fe1424688aed206142611df [term 1 LEADER]: Becoming Leader. State: Replica: a980bf644fe1424688aed206142611df, State: Running, Role: LEADER
I20260925 07:25:20.280292 8074 consensus_queue.cc:237] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P a980bf644fe1424688aed206142611df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } } peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } }
I20260925 07:25:20.280880 7474 catalog_manager.cc:5495] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P a980bf644fe1424688aed206142611df reported cstate change: term changed from 0 to 1, leader changed from <none> to a980bf644fe1424688aed206142611df (127.5.158.67). New cstate: current_term: 1 leader_uuid: "a980bf644fe1424688aed206142611df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a980bf644fe1424688aed206142611df" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 34023 } health_report { overall_health: HEALTHY } } }
I20260925 07:25:20.293401 8090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:20.291526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60880 (local address 127.5.158.67:34023)
0925 07:25:20.291652 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:20.291653 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:20.291658 (+ 5us) server_negotiation.cc:407] Connection header received
0925 07:25:20.291683 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:20.291683 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:20.291696 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:20.291753 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 07:25:20.291835 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:20.292213 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:20.292761 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:20.292912 (+ 151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:20.292952 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:20.293057 (+ 105us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 07:25:20.293223 (+ 166us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 07:25:20.293276 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:20.293310 (+ 34us) server_negotiation.cc:299] Negotiation successful
0925 07:25:20.293322 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":77,"threads_started":1}
I20260925 07:25:20.297258 8078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:20.295334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:47543 (local address 127.5.158.65:42225)
0925 07:25:20.295368 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:20.295369 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:20.295490 (+ 121us) server_negotiation.cc:407] Connection header received
0925 07:25:20.295522 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:20.295522 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:20.295531 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:20.295563 (+ 32us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 07:25:20.295630 (+ 67us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:20.296012 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:20.296687 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:20.296852 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:20.296897 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:20.297056 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 07:25:20.297077 (+ 21us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 07:25:20.297154 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:20.297202 (+ 48us) server_negotiation.cc:299] Negotiation successful
0925 07:25:20.297218 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260925 07:25:20.297477 8093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:20.295290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:42225 (local address 127.5.158.67:47543)
0925 07:25:20.295433 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:20.295441 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:20.295486 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:20.295564 (+ 78us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:20.295564 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:20.295565 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 07:25:20.295598 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:20.295599 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:20.296062 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:20.296063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:20.296621 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:20.296622 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:20.296927 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:20.296928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:20.296969 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:20.296980 (+ 11us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 07:25:20.297166 (+ 186us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 07:25:20.297168 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 07:25:20.297199 (+ 31us) client_negotiation.cc:240] Negotiation successful
0925 07:25:20.297229 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":65,"threads_started":1}
I20260925 07:25:20.299242 7632 raft_consensus.cc:1261] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P 9340b3d9a5314d24bdde11845526f19d [term 1 FOLLOWER]: Refusing update from remote peer a980bf644fe1424688aed206142611df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:25:20.299240 7771 raft_consensus.cc:1261] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P 6b563430b1a945acb4bf3ea743cbca17 [term 1 FOLLOWER]: Refusing update from remote peer a980bf644fe1424688aed206142611df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:25:20.299479 8074 consensus_queue.cc:1035] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P a980bf644fe1424688aed206142611df [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b563430b1a945acb4bf3ea743cbca17" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 35403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:25:20.299589 8088 consensus_queue.cc:1035] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P a980bf644fe1424688aed206142611df [LEADER]: Connected to new peer: Peer: permanent_uuid: "9340b3d9a5314d24bdde11845526f19d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:25:20.306293 8090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:20.303679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:53611 (local address 127.5.158.67:34023)
0925 07:25:20.303707 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:20.303709 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:20.303795 (+ 86us) server_negotiation.cc:407] Connection header received
0925 07:25:20.303855 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:20.303856 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:20.303864 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:20.303914 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 07:25:20.304027 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:20.304569 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:20.305446 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:20.305724 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:20.305786 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:20.305964 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 07:25:20.306037 (+ 73us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 07:25:20.306092 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:20.306219 (+ 127us) server_negotiation.cc:299] Negotiation successful
0925 07:25:20.306238 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260925 07:25:20.306296 8097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:20.303542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.67:34023 (local address 127.5.158.65:53611)
0925 07:25:20.303741 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:20.303750 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:20.303807 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:20.303916 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:20.303917 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:20.303918 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 07:25:20.303953 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:20.303954 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:20.304726 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:20.304727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:20.305390 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:20.305392 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:20.305822 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:20.305823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:20.305878 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:20.305881 (+ 3us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 07:25:20.306138 (+ 257us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 07:25:20.306140 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 07:25:20.306179 (+ 39us) client_negotiation.cc:240] Negotiation successful
0925 07:25:20.306225 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":68,"threads_started":1}
W20260925 07:25:20.306625 8041 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260925 07:25:21.823421 7473 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45960:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 07:25:21.823525 7473 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45960:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 07:25:21.824237 7473 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 39542b3be6b7448ab00f36a31d0c7915: Sending DeleteTablet for 3 replicas of tablet 5ae0fc3a2a5b4cdd8ee0a469b9a72144
I20260925 07:25:21.824460 7612 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ae0fc3a2a5b4cdd8ee0a469b9a72144 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:25:21 UTC) from {username='kudu', principal='kudu/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:46538
I20260925 07:25:21.824467 7752 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ae0fc3a2a5b4cdd8ee0a469b9a72144 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:25:21 UTC) from {username='kudu', principal='kudu/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:47972
I20260925 07:25:21.824559 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 7547
I20260925 07:25:21.825073 8122 tablet_replica.cc:330] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P 6b563430b1a945acb4bf3ea743cbca17: stopping tablet replica
I20260925 07:25:21.825145 8122 raft_consensus.cc:2229] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P 6b563430b1a945acb4bf3ea743cbca17 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 07:25:21.825176 8122 raft_consensus.cc:2258] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P 6b563430b1a945acb4bf3ea743cbca17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:25:21.825374 8122 ts_tablet_manager.cc:1905] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P 6b563430b1a945acb4bf3ea743cbca17: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:25:21.826074 7892 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ae0fc3a2a5b4cdd8ee0a469b9a72144 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:25:21 UTC) from {username='kudu', principal='kudu/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:60872
I20260925 07:25:21.826259 8123 tablet_replica.cc:330] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P a980bf644fe1424688aed206142611df: stopping tablet replica
I20260925 07:25:21.826320 8123 raft_consensus.cc:2229] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P a980bf644fe1424688aed206142611df [term 1 LEADER]: Raft consensus shutting down.
I20260925 07:25:21.826401 8123 raft_consensus.cc:2258] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P a980bf644fe1424688aed206142611df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:25:21.826594 8123 ts_tablet_manager.cc:1905] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P a980bf644fe1424688aed206142611df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:25:21.826814 8122 ts_tablet_manager.cc:1918] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P 6b563430b1a945acb4bf3ea743cbca17: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 07:25:21.826908 8122 log.cc:1196] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P 6b563430b1a945acb4bf3ea743cbca17: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-1/wal/wals/5ae0fc3a2a5b4cdd8ee0a469b9a72144
I20260925 07:25:21.827145 8122 ts_tablet_manager.cc:1939] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P 6b563430b1a945acb4bf3ea743cbca17: Deleting consensus metadata
I20260925 07:25:21.827459 7461 catalog_manager.cc:4841] TS 6b563430b1a945acb4bf3ea743cbca17 (127.5.158.66:35403): tablet 5ae0fc3a2a5b4cdd8ee0a469b9a72144 (table test-table [id=043d8cd6ee264f2a887f0a37121e3bce]) successfully deleted
I20260925 07:25:21.827996 8123 ts_tablet_manager.cc:1918] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P a980bf644fe1424688aed206142611df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 07:25:21.828061 8123 log.cc:1196] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P a980bf644fe1424688aed206142611df: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790321110323796-5753-0/minicluster-data/ts-2/wal/wals/5ae0fc3a2a5b4cdd8ee0a469b9a72144
I20260925 07:25:21.828253 8123 ts_tablet_manager.cc:1939] T 5ae0fc3a2a5b4cdd8ee0a469b9a72144 P a980bf644fe1424688aed206142611df: Deleting consensus metadata
I20260925 07:25:21.828544 7462 catalog_manager.cc:4841] TS a980bf644fe1424688aed206142611df (127.5.158.67:34023): tablet 5ae0fc3a2a5b4cdd8ee0a469b9a72144 (table test-table [id=043d8cd6ee264f2a887f0a37121e3bce]) successfully deleted
W20260925 07:25:21.831314 7461 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.158.65:42225 (error 108)
I20260925 07:25:21.831573 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 7687
I20260925 07:25:21.836742 8125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:21.831645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:42225 (local address 127.0.0.1:46582)
0925 07:25:21.836643 (+ 4998us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:21.836696 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.65:42225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4969,"thread_start_us":53,"threads_started":1}
W20260925 07:25:21.836874 7461 catalog_manager.cc:4584] TS 9340b3d9a5314d24bdde11845526f19d (127.5.158.65:42225): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5ae0fc3a2a5b4cdd8ee0a469b9a72144: Network error: Client connection negotiation failed: client connection to 127.5.158.65:42225: connect: Connection refused (error 111)
I20260925 07:25:21.836993 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 7827
I20260925 07:25:21.841902 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 7443
2026-09-25T07:25:21Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5869 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 07:25:21 dist-test-slave-1drq krb5kdc[8128](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 07:25:21 dist-test-slave-1drq krb5kdc[8128](info): set up 2 sockets
Sep 25 07:25:21 dist-test-slave-1drq krb5kdc[8128](info): commencing operation
krb5kdc: starting...
W20260925 07:25:22.784416 5753 mini_kdc.cc:121] Time spent starting KDC: real 0.923s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Sep 25 07:25:22 dist-test-slave-1drq krb5kdc[8128](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321122, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T07:25:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T07:25:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
I20260925 07:25:22.885552 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:45071
--webserver_interface=127.5.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:43643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.158.126:45071
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:25:22.955379 8141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:25:22.955559 8141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:25:22.955590 8141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:25:22.956619 8141 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:25:22.956674 8141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:25:22.956701 8141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:25:22.956728 8141 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:25:22.956768 8141 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:25:22.958226 8141 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:43643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.158.126:45071
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:45071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8141
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:25:22.958483 8141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:25:22.958657 8141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:25:22.962798 8149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:22.962890 8141 server_base.cc:1034] running on GCE node
W20260925 07:25:22.962810 8147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:22.962806 8146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:22.963186 8141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:25:22.963372 8141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:25:22.964543 8141 hybrid_clock.cc:648] HybridClock initialized: now 1790321122964533 us; error 25 us; skew 500 ppm
Sep 25 07:25:22 dist-test-slave-1drq krb5kdc[8128](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321122, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:25:22.966696 8141 init.cc:377] Logged in from keytab as kudu/127.5.158.126@KRBTEST.COM (short username kudu)
I20260925 07:25:22.967118 8141 webserver.cc:460] Webserver started at http://127.5.158.126:39269/ using document root <none> and password file <none>
I20260925 07:25:22.967285 8141 fs_manager.cc:359] Metadata directory not provided
I20260925 07:25:22.967322 8141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:25:22.967406 8141 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:25:22.968258 8141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/master-0/data/instance:
uuid: "2a21130716a445e1bb34ce88042713bf"
format_stamp: "Formatted at 2026-09-25 07:25:22 on dist-test-slave-1drq"
server_key: "6f77d027f516f1da4731a5633368f57b"
server_key_iv: "407370ecef60840472f965fdd7c0f851"
server_key_version: "encryptionkey@0"
I20260925 07:25:22.968498 8141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/master-0/wal/instance:
uuid: "2a21130716a445e1bb34ce88042713bf"
format_stamp: "Formatted at 2026-09-25 07:25:22 on dist-test-slave-1drq"
server_key: "6f77d027f516f1da4731a5633368f57b"
server_key_iv: "407370ecef60840472f965fdd7c0f851"
server_key_version: "encryptionkey@0"
I20260925 07:25:22.969561 8141 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 07:25:22.970112 8156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:22.970274 8141 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:25:22.970342 8141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "2a21130716a445e1bb34ce88042713bf"
format_stamp: "Formatted at 2026-09-25 07:25:22 on dist-test-slave-1drq"
server_key: "6f77d027f516f1da4731a5633368f57b"
server_key_iv: "407370ecef60840472f965fdd7c0f851"
server_key_version: "encryptionkey@0"
I20260925 07:25:22.970386 8141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:25:22.978679 8141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:25:22.980152 8141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:25:22.980288 8141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:25:22.983894 8141 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.126:45071
I20260925 07:25:22.983942 8208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.126:45071 every 8 connection(s)
I20260925 07:25:22.984208 8141 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:25:22.984918 8209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:25:22.986729 8209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a21130716a445e1bb34ce88042713bf: Bootstrap starting.
I20260925 07:25:22.987212 8209 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a21130716a445e1bb34ce88042713bf: Neither blocks nor log segments found. Creating new log.
I20260925 07:25:22.987432 8209 log.cc:824] T 00000000000000000000000000000000 P 2a21130716a445e1bb34ce88042713bf: Log is configured to *not* fsync() on all Append() calls
I20260925 07:25:22.988003 8209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a21130716a445e1bb34ce88042713bf: No bootstrap required, opened a new log
I20260925 07:25:22.989115 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 8141
I20260925 07:25:22.989208 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/master-0/wal/instance
I20260925 07:25:22.989380 5753 external_mini_cluster.cc:1428] Setting key 455dfa0ddf3cdbf06d1b8f491942df51
I20260925 07:25:22.989879 8209 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a21130716a445e1bb34ce88042713bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a21130716a445e1bb34ce88042713bf" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 45071 } }
I20260925 07:25:22.990059 8209 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2a21130716a445e1bb34ce88042713bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:25:22.990104 8209 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a21130716a445e1bb34ce88042713bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a21130716a445e1bb34ce88042713bf, State: Initialized, Role: FOLLOWER
I20260925 07:25:22.990212 8209 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a21130716a445e1bb34ce88042713bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a21130716a445e1bb34ce88042713bf" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 45071 } }
I20260925 07:25:22.990281 8209 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2a21130716a445e1bb34ce88042713bf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 07:25:22.990311 8209 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a21130716a445e1bb34ce88042713bf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 07:25:22.990381 8209 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a21130716a445e1bb34ce88042713bf [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:25:22.991276 8209 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a21130716a445e1bb34ce88042713bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a21130716a445e1bb34ce88042713bf" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 45071 } }
I20260925 07:25:22.991350 8209 leader_election.cc:302] T 00000000000000000000000000000000 P 2a21130716a445e1bb34ce88042713bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a21130716a445e1bb34ce88042713bf; no voters:
I20260925 07:25:22.991525 8209 leader_election.cc:288] T 00000000000000000000000000000000 P 2a21130716a445e1bb34ce88042713bf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 07:25:22.991552 8214 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a21130716a445e1bb34ce88042713bf [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:25:22.991643 8214 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2a21130716a445e1bb34ce88042713bf [term 1 LEADER]: Becoming Leader. State: Replica: 2a21130716a445e1bb34ce88042713bf, State: Running, Role: LEADER
I20260925 07:25:22.991714 8214 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2a21130716a445e1bb34ce88042713bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a21130716a445e1bb34ce88042713bf" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 45071 } }
I20260925 07:25:22.991750 8209 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a21130716a445e1bb34ce88042713bf [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:25:22.992180 8215 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a21130716a445e1bb34ce88042713bf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2a21130716a445e1bb34ce88042713bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a21130716a445e1bb34ce88042713bf" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 45071 } } }
I20260925 07:25:22.992396 8215 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a21130716a445e1bb34ce88042713bf [sys.catalog]: This master's current role is: LEADER
I20260925 07:25:22.992185 8216 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a21130716a445e1bb34ce88042713bf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a21130716a445e1bb34ce88042713bf. Latest consensus state: current_term: 1 leader_uuid: "2a21130716a445e1bb34ce88042713bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a21130716a445e1bb34ce88042713bf" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 45071 } } }
I20260925 07:25:22.992625 8216 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a21130716a445e1bb34ce88042713bf [sys.catalog]: This master's current role is: LEADER
I20260925 07:25:22.992761 8223 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 07:25:22.993201 8223 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 25 07:25:22 dist-test-slave-1drq krb5kdc[8128](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321122, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:25:22.997555 8223 catalog_manager.cc:1347] Generated new cluster ID: f2c8bd030e8f4353a4fbb042e2ab0d28
I20260925 07:25:22.997603 8223 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 07:25:23.003079 8213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:22.989829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51354 (local address 127.5.158.126:45071)
0925 07:25:22.989981 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:22.989994 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:22.990014 (+ 20us) server_negotiation.cc:407] Connection header received
0925 07:25:22.990110 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:22.990116 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:22.990164 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:22.990297 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:22.991010 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:22.992221 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:22.993301 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:22.993906 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:22.994005 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:22.996287 (+ 2282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:22.996290 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:22.996290 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:22.996300 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:23.001625 (+ 5325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:23.001968 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:23.001971 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:23.001971 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:23.002058 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:23.002257 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:23.002258 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:23.002258 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:23.002464 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:23.002608 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:23.002727 (+ 119us) server_negotiation.cc:299] Negotiation successful
0925 07:25:23.002937 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":85,"threads_started":1}
I20260925 07:25:23.016615 8223 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 07:25:23.017213 8223 catalog_manager.cc:1504] Loading token signing keys...
I20260925 07:25:23.025957 8223 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2a21130716a445e1bb34ce88042713bf: Generated new TSK 0
I20260925 07:25:23.026108 8223 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.158.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab.
Entry for principal kudu/127.5.158.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab.
I20260925 07:25:23.057796 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.65:0
--local_ip_for_outbound_sockets=127.5.158.65
--webserver_interface=127.5.158.65
--webserver_port=0
--tserver_master_addrs=127.5.158.126:45071
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab
--principal=kudu/127.5.158.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:43643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:25:23.134146 8235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:25:23.134322 8235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:25:23.134342 8235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:25:23.135329 8235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:25:23.135366 8235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:25:23.135413 8235 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.65
I20260925 07:25:23.136823 8235 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:43643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab
--principal=kudu/127.5.158.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.158.65
--webserver_port=0
--tserver_master_addrs=127.5.158.126:45071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8235
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.65
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:25:23.137053 8235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:25:23.137284 8235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 07:25:23.142004 8235 server_base.cc:1034] running on GCE node
W20260925 07:25:23.142004 8243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:23.142189 8240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:23.142238 8241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:23.142463 8235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:25:23.142637 8235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:25:23.143803 8235 hybrid_clock.cc:648] HybridClock initialized: now 1790321123143784 us; error 37 us; skew 500 ppm
Sep 25 07:25:23 dist-test-slave-1drq krb5kdc[8128](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321123, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:25:23.146167 8235 init.cc:377] Logged in from keytab as kudu/127.5.158.65@KRBTEST.COM (short username kudu)
I20260925 07:25:23.146554 8235 webserver.cc:460] Webserver started at http://127.5.158.65:41159/ using document root <none> and password file <none>
I20260925 07:25:23.146761 8235 fs_manager.cc:359] Metadata directory not provided
I20260925 07:25:23.146803 8235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:25:23.146935 8235 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:25:23.147842 8235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-0/data/instance:
uuid: "a3437ea1fe694a118f97b9121ca6964f"
format_stamp: "Formatted at 2026-09-25 07:25:23 on dist-test-slave-1drq"
server_key: "c1cfd53e78120ae28ca4b2c7e6ce99ca"
server_key_iv: "9f6cbdd1ada43c8c78c5d3b8fe21e77e"
server_key_version: "encryptionkey@0"
I20260925 07:25:23.148180 8235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-0/wal/instance:
uuid: "a3437ea1fe694a118f97b9121ca6964f"
format_stamp: "Formatted at 2026-09-25 07:25:23 on dist-test-slave-1drq"
server_key: "c1cfd53e78120ae28ca4b2c7e6ce99ca"
server_key_iv: "9f6cbdd1ada43c8c78c5d3b8fe21e77e"
server_key_version: "encryptionkey@0"
I20260925 07:25:23.149308 8235 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 07:25:23.149962 8250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:23.150137 8235 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:25:23.150218 8235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-0/wal
uuid: "a3437ea1fe694a118f97b9121ca6964f"
format_stamp: "Formatted at 2026-09-25 07:25:23 on dist-test-slave-1drq"
server_key: "c1cfd53e78120ae28ca4b2c7e6ce99ca"
server_key_iv: "9f6cbdd1ada43c8c78c5d3b8fe21e77e"
server_key_version: "encryptionkey@0"
I20260925 07:25:23.150283 8235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:25:23.168601 8235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:25:23.170153 8235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:25:23.170274 8235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:25:23.170459 8235 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:25:23.170709 8235 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:25:23.170763 8235 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:23.170797 8235 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:25:23.170823 8235 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:23.176167 8235 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.65:33741
I20260925 07:25:23.176265 8363 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.65:33741 every 8 connection(s)
I20260925 07:25:23.176504 8235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
Sep 25 07:25:23 dist-test-slave-1drq krb5kdc[8128](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321123, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.65@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:25:23.182229 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 8235
I20260925 07:25:23.182314 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-0/wal/instance
I20260925 07:25:23.182461 5753 external_mini_cluster.cc:1428] Setting key ebe5ff14523820c8a68e98edcce4b3e0
I20260925 07:25:23.185043 8213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:23.177198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:51863 (local address 127.5.158.126:45071)
0925 07:25:23.177262 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:23.177264 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:23.177461 (+ 197us) server_negotiation.cc:407] Connection header received
0925 07:25:23.177929 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:23.177930 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:23.177942 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:23.178023 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:23.179043 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:23.179557 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:23.180491 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:23.180685 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:23.180753 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:23.182621 (+ 1868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:23.182622 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:23.182623 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:23.182628 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:23.183810 (+ 1182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:23.184230 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:23.184231 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:23.184231 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:23.184280 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:23.184520 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:23.184521 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:23.184521 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:23.184608 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:23.184652 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:23.184925 (+ 273us) server_negotiation.cc:299] Negotiation successful
0925 07:25:23.184991 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 07:25:23.185292 8366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:23.177212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:45071 (local address 127.5.158.65:51863)
0925 07:25:23.177347 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:23.177378 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:23.177814 (+ 436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:23.178098 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:23.178101 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:23.178678 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:23.178848 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:23.178852 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:23.179656 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:23.179657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:23.180404 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:23.180406 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:23.180760 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:23.180761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:23.180867 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:23.181319 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:23.181332 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:23.182490 (+ 1158us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:23.183892 (+ 1402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:23.183897 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:23.183899 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:23.184158 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:23.184343 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:23.184344 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:23.184344 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:23.184463 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:23.184663 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:23.184668 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:23.184803 (+ 135us) client_negotiation.cc:769] Sending connection context
0925 07:25:23.184907 (+ 104us) client_negotiation.cc:240] Negotiation successful
0925 07:25:23.185045 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":68,"threads_started":1}
I20260925 07:25:23.185753 8364 heartbeater.cc:347] Connected to a master server at 127.5.158.126:45071
I20260925 07:25:23.185832 8364 heartbeater.cc:464] Registering TS with master...
I20260925 07:25:23.185972 8364 heartbeater.cc:511] Master 127.5.158.126:45071 requested a full tablet report, sending...
I20260925 07:25:23.186420 8173 ts_manager.cc:194] Registered new tserver with Master: a3437ea1fe694a118f97b9121ca6964f (127.5.158.65:33741)
I20260925 07:25:23.187008 8173 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.65@KRBTEST.COM'} at 127.5.158.65:51863
WARNING: no policy specified for kudu/127.5.158.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab.
Entry for principal kudu/127.5.158.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab.
I20260925 07:25:23.203704 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.66:0
--local_ip_for_outbound_sockets=127.5.158.66
--webserver_interface=127.5.158.66
--webserver_port=0
--tserver_master_addrs=127.5.158.126:45071
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab
--principal=kudu/127.5.158.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:43643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:25:23.273375 8369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:25:23.273527 8369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:25:23.273547 8369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:25:23.274534 8369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:25:23.274570 8369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:25:23.274613 8369 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.66
I20260925 07:25:23.275771 8369 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:43643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab
--principal=kudu/127.5.158.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.158.66
--webserver_port=0
--tserver_master_addrs=127.5.158.126:45071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8369
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.66
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:25:23.275961 8369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:25:23.276181 8369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:25:23.280467 8375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:23.280479 8374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:23.280478 8377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:23.280490 8369 server_base.cc:1034] running on GCE node
I20260925 07:25:23.280788 8369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:25:23.281026 8369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:25:23.282219 8369 hybrid_clock.cc:648] HybridClock initialized: now 1790321123282215 us; error 53 us; skew 500 ppm
Sep 25 07:25:23 dist-test-slave-1drq krb5kdc[8128](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321123, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:25:23.284214 8369 init.cc:377] Logged in from keytab as kudu/127.5.158.66@KRBTEST.COM (short username kudu)
I20260925 07:25:23.284602 8369 webserver.cc:460] Webserver started at http://127.5.158.66:33689/ using document root <none> and password file <none>
I20260925 07:25:23.284789 8369 fs_manager.cc:359] Metadata directory not provided
I20260925 07:25:23.284845 8369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:25:23.284981 8369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:25:23.285858 8369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-1/data/instance:
uuid: "14b003c2bad043739a735d9b82f02ea5"
format_stamp: "Formatted at 2026-09-25 07:25:23 on dist-test-slave-1drq"
server_key: "00ef626cbc50673d3a16760074b1f803"
server_key_iv: "f21bfda17b8be4aefc0162eeacea7e31"
server_key_version: "encryptionkey@0"
I20260925 07:25:23.286162 8369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-1/wal/instance:
uuid: "14b003c2bad043739a735d9b82f02ea5"
format_stamp: "Formatted at 2026-09-25 07:25:23 on dist-test-slave-1drq"
server_key: "00ef626cbc50673d3a16760074b1f803"
server_key_iv: "f21bfda17b8be4aefc0162eeacea7e31"
server_key_version: "encryptionkey@0"
I20260925 07:25:23.287307 8369 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260925 07:25:23.288002 8384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:23.288147 8369 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:25:23.288232 8369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-1/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-1/wal
uuid: "14b003c2bad043739a735d9b82f02ea5"
format_stamp: "Formatted at 2026-09-25 07:25:23 on dist-test-slave-1drq"
server_key: "00ef626cbc50673d3a16760074b1f803"
server_key_iv: "f21bfda17b8be4aefc0162eeacea7e31"
server_key_version: "encryptionkey@0"
I20260925 07:25:23.288306 8369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:25:23.307895 8369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:25:23.309887 8369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:25:23.310034 8369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:25:23.310253 8369 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:25:23.310557 8369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:25:23.310611 8369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:23.310649 8369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:25:23.310678 8369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:23.316810 8369 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.66:36933
I20260925 07:25:23.316903 8497 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.66:36933 every 8 connection(s)
I20260925 07:25:23.317296 8369 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
Sep 25 07:25:23 dist-test-slave-1drq krb5kdc[8128](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321123, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.66@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:25:23.325340 8213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:23.318024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:54979 (local address 127.5.158.126:45071)
0925 07:25:23.318069 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:23.318071 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:23.318219 (+ 148us) server_negotiation.cc:407] Connection header received
0925 07:25:23.318673 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:23.318673 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:23.318684 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:23.318757 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:23.319569 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:23.320070 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:23.320935 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:23.321201 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:23.321256 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:23.323068 (+ 1812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:23.323069 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:23.323069 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:23.323075 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:23.324260 (+ 1185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:23.324574 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:23.324575 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:23.324576 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:23.324627 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:23.324811 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:23.324812 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:23.324812 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:23.324905 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:23.324955 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:23.325203 (+ 248us) server_negotiation.cc:299] Negotiation successful
0925 07:25:23.325265 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 07:25:23.325453 8500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:23.317974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:45071 (local address 127.5.158.66:54979)
0925 07:25:23.318117 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:23.318155 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:23.318594 (+ 439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:23.318807 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:23.318810 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:23.319325 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:23.319498 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:23.319502 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:23.320141 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:23.320142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:23.320859 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:23.320860 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:23.321280 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:23.321281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:23.321373 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:23.321830 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:23.321837 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:23.322989 (+ 1152us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:23.324328 (+ 1339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:23.324333 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:23.324334 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:23.324514 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:23.324680 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:23.324681 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:23.324681 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:23.324761 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:23.324956 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:23.324959 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:23.325047 (+ 88us) client_negotiation.cc:769] Sending connection context
0925 07:25:23.325174 (+ 127us) client_negotiation.cc:240] Negotiation successful
0925 07:25:23.325270 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":82,"threads_started":1}
I20260925 07:25:23.325894 8498 heartbeater.cc:347] Connected to a master server at 127.5.158.126:45071
I20260925 07:25:23.325992 8498 heartbeater.cc:464] Registering TS with master...
I20260925 07:25:23.326149 8498 heartbeater.cc:511] Master 127.5.158.126:45071 requested a full tablet report, sending...
I20260925 07:25:23.326524 8173 ts_manager.cc:194] Registered new tserver with Master: 14b003c2bad043739a735d9b82f02ea5 (127.5.158.66:36933)
I20260925 07:25:23.326908 8173 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.66@KRBTEST.COM'} at 127.5.158.66:54979
I20260925 07:25:23.327348 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 8369
I20260925 07:25:23.327453 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-1/wal/instance
I20260925 07:25:23.327633 5753 external_mini_cluster.cc:1428] Setting key 2ac54846967a4d17103c5c2a5e9bd229
WARNING: no policy specified for kudu/127.5.158.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab.
Entry for principal kudu/127.5.158.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab.
I20260925 07:25:23.351223 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.67:0
--local_ip_for_outbound_sockets=127.5.158.67
--webserver_interface=127.5.158.67
--webserver_port=0
--tserver_master_addrs=127.5.158.126:45071
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab
--principal=kudu/127.5.158.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:43643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:25:23.423020 8503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:25:23.423204 8503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:25:23.423235 8503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:25:23.424268 8503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:25:23.424324 8503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:25:23.424364 8503 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.67
I20260925 07:25:23.425622 8503 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:43643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab
--principal=kudu/127.5.158.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.158.67
--webserver_port=0
--tserver_master_addrs=127.5.158.126:45071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8503
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.67
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:25:23.425858 8503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:25:23.426050 8503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:25:23.430542 8509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:23.430550 8508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:23.430557 8511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:23.430748 8503 server_base.cc:1034] running on GCE node
I20260925 07:25:23.430943 8503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:25:23.431103 8503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:25:23.432252 8503 hybrid_clock.cc:648] HybridClock initialized: now 1790321123432225 us; error 54 us; skew 500 ppm
Sep 25 07:25:23 dist-test-slave-1drq krb5kdc[8128](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321123, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:25:23.434445 8503 init.cc:377] Logged in from keytab as kudu/127.5.158.67@KRBTEST.COM (short username kudu)
I20260925 07:25:23.434821 8503 webserver.cc:460] Webserver started at http://127.5.158.67:43525/ using document root <none> and password file <none>
I20260925 07:25:23.434974 8503 fs_manager.cc:359] Metadata directory not provided
I20260925 07:25:23.435007 8503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:25:23.435077 8503 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:25:23.435846 8503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-2/data/instance:
uuid: "79b0280e934741c6b7048829debca080"
format_stamp: "Formatted at 2026-09-25 07:25:23 on dist-test-slave-1drq"
server_key: "37de22bcd1f46938b3c4098a0075eda6"
server_key_iv: "e4b02daf3dae6a0aec6fce0262dd8312"
server_key_version: "encryptionkey@0"
I20260925 07:25:23.436126 8503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-2/wal/instance:
uuid: "79b0280e934741c6b7048829debca080"
format_stamp: "Formatted at 2026-09-25 07:25:23 on dist-test-slave-1drq"
server_key: "37de22bcd1f46938b3c4098a0075eda6"
server_key_iv: "e4b02daf3dae6a0aec6fce0262dd8312"
server_key_version: "encryptionkey@0"
I20260925 07:25:23.437268 8503 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 07:25:23.437896 8518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:23.438072 8503 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:25:23.438154 8503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-2/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-2/wal
uuid: "79b0280e934741c6b7048829debca080"
format_stamp: "Formatted at 2026-09-25 07:25:23 on dist-test-slave-1drq"
server_key: "37de22bcd1f46938b3c4098a0075eda6"
server_key_iv: "e4b02daf3dae6a0aec6fce0262dd8312"
server_key_version: "encryptionkey@0"
I20260925 07:25:23.438200 8503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:25:23.453327 8503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:25:23.454826 8503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:25:23.454954 8503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:25:23.455116 8503 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:25:23.455363 8503 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:25:23.455391 8503 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:23.455410 8503 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:25:23.455462 8503 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:23.460620 8503 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.67:44511
I20260925 07:25:23.460671 8631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.67:44511 every 8 connection(s)
I20260925 07:25:23.460932 8503 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
I20260925 07:25:23.465216 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 8503
I20260925 07:25:23.465318 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-2/wal/instance
I20260925 07:25:23.465502 5753 external_mini_cluster.cc:1428] Setting key 1df40896fbde431299ee23a02a5fc78c
Sep 25 07:25:23 dist-test-slave-1drq krb5kdc[8128](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321123, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.67@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:25:23.469409 8213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:23.461545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:53403 (local address 127.5.158.126:45071)
0925 07:25:23.461583 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:23.461585 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:23.461733 (+ 148us) server_negotiation.cc:407] Connection header received
0925 07:25:23.462120 (+ 387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:23.462121 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:23.462131 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:23.462170 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:23.462942 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:23.463417 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:23.464254 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:23.464409 (+ 155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:23.464458 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:23.466985 (+ 2527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:23.466986 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:23.466986 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:23.466992 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:23.468189 (+ 1197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:23.468573 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:23.468573 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:23.468574 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:23.468617 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:23.468852 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:23.468852 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:23.468853 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:23.468938 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:23.468984 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:23.469296 (+ 312us) server_negotiation.cc:299] Negotiation successful
0925 07:25:23.469350 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 07:25:23.469667 8634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:23.461525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:45071 (local address 127.5.158.67:53403)
0925 07:25:23.461646 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:23.461672 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:23.462049 (+ 377us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:23.462219 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:23.462222 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:23.462713 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:23.462877 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:23.462881 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:23.463493 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:23.463494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:23.464189 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:23.464190 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:23.464605 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:23.464606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:23.464736 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:23.465369 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:23.465380 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:23.466906 (+ 1526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:23.468278 (+ 1372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:23.468284 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:23.468285 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:23.468509 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:23.468693 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:23.468693 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:23.468694 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:23.468796 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:23.468994 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:23.468998 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:23.469172 (+ 174us) client_negotiation.cc:769] Sending connection context
0925 07:25:23.469278 (+ 106us) client_negotiation.cc:240] Negotiation successful
0925 07:25:23.469397 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":72,"threads_started":1}
I20260925 07:25:23.470111 8632 heartbeater.cc:347] Connected to a master server at 127.5.158.126:45071
I20260925 07:25:23.470234 8632 heartbeater.cc:464] Registering TS with master...
I20260925 07:25:23.470376 8632 heartbeater.cc:511] Master 127.5.158.126:45071 requested a full tablet report, sending...
I20260925 07:25:23.470662 8173 ts_manager.cc:194] Registered new tserver with Master: 79b0280e934741c6b7048829debca080 (127.5.158.67:44511)
I20260925 07:25:23.471009 8173 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.67@KRBTEST.COM'} at 127.5.158.67:53403
I20260925 07:25:23.477054 5753 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260925 07:25:23.486696 8213 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:25:23.485913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51364 (local address 127.5.158.126:45071)
0925 07:25:23.485961 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:23.485963 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:23.486064 (+ 101us) server_negotiation.cc:407] Connection header received
0925 07:25:23.486140 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:23.486140 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:23.486149 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:23.486207 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:23.486644 (+ 437us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51364: BlockingRecv error: recv got EOF from 127.0.0.1:51364 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
I20260925 07:25:23.487555 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 8235
I20260925 07:25:23.491698 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 8369
I20260925 07:25:23.495824 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 8503
I20260925 07:25:23.499471 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 8141
2026-09-25T07:25:23Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1657 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790321110323796-5753-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 07:25:23 dist-test-slave-1drq krb5kdc[8644](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 07:25:23 dist-test-slave-1drq krb5kdc[8644](info): set up 2 sockets
Sep 25 07:25:23 dist-test-slave-1drq krb5kdc[8644](info): commencing operation
krb5kdc: starting...
W20260925 07:25:24.444782 5753 mini_kdc.cc:121] Time spent starting KDC: real 0.926s user 0.003s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Sep 25 07:25:24 dist-test-slave-1drq krb5kdc[8644](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321124, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T07:25:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T07:25:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
I20260925 07:25:24.549022 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:41739
--webserver_interface=127.5.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:44619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.158.126:41739
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:25:24.620929 8657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:25:24.621122 8657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:25:24.621145 8657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:25:24.622225 8657 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:25:24.622262 8657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:25:24.622274 8657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:25:24.622285 8657 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:25:24.622296 8657 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:25:24.623528 8657 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:44619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790321110323796-5753-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.158.126:41739
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:41739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8657
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:25:24.623679 8657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:25:24.623824 8657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:25:24.628150 8663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:24.628158 8662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:24.628175 8665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:24.628407 8657 server_base.cc:1034] running on GCE node
I20260925 07:25:24.628654 8657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:25:24.628911 8657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:25:24.630087 8657 hybrid_clock.cc:648] HybridClock initialized: now 1790321124630052 us; error 42 us; skew 500 ppm
Sep 25 07:25:24 dist-test-slave-1drq krb5kdc[8644](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321124, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:25:24.632256 8657 init.cc:377] Logged in from keytab as kudu/127.5.158.126@KRBTEST.COM (short username kudu)
I20260925 07:25:24.632681 8657 webserver.cc:460] Webserver started at http://127.5.158.126:40817/ using document root <none> and password file <none>
I20260925 07:25:24.632891 8657 fs_manager.cc:359] Metadata directory not provided
I20260925 07:25:24.632957 8657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:25:24.633076 8657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:25:24.633996 8657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790321110323796-5753-0/minicluster-data/master-0/data/instance:
uuid: "14944af41599467eb39a5490a6d3d67d"
format_stamp: "Formatted at 2026-09-25 07:25:24 on dist-test-slave-1drq"
server_key: "a8bb90b1914607f294ec3d1090b615f8"
server_key_iv: "f29a2f74b62f390e088fbd8dde6ac37c"
server_key_version: "encryptionkey@0"
I20260925 07:25:24.634328 8657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790321110323796-5753-0/minicluster-data/master-0/wal/instance:
uuid: "14944af41599467eb39a5490a6d3d67d"
format_stamp: "Formatted at 2026-09-25 07:25:24 on dist-test-slave-1drq"
server_key: "a8bb90b1914607f294ec3d1090b615f8"
server_key_iv: "f29a2f74b62f390e088fbd8dde6ac37c"
server_key_version: "encryptionkey@0"
I20260925 07:25:24.635470 8657 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 07:25:24.636101 8672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:24.636252 8657 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 07:25:24.636339 8657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "14944af41599467eb39a5490a6d3d67d"
format_stamp: "Formatted at 2026-09-25 07:25:24 on dist-test-slave-1drq"
server_key: "a8bb90b1914607f294ec3d1090b615f8"
server_key_iv: "f29a2f74b62f390e088fbd8dde6ac37c"
server_key_version: "encryptionkey@0"
I20260925 07:25:24.636415 8657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:25:24.651320 8657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:25:24.652880 8657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:25:24.653014 8657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:25:24.656580 8657 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.126:41739
I20260925 07:25:24.656618 8724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.126:41739 every 8 connection(s)
I20260925 07:25:24.656915 8657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:25:24.657776 8725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:25:24.659663 8725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14944af41599467eb39a5490a6d3d67d: Bootstrap starting.
I20260925 07:25:24.660181 8725 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 14944af41599467eb39a5490a6d3d67d: Neither blocks nor log segments found. Creating new log.
I20260925 07:25:24.660447 8725 log.cc:824] T 00000000000000000000000000000000 P 14944af41599467eb39a5490a6d3d67d: Log is configured to *not* fsync() on all Append() calls
I20260925 07:25:24.661011 8725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14944af41599467eb39a5490a6d3d67d: No bootstrap required, opened a new log
I20260925 07:25:24.662395 8725 raft_consensus.cc:348] T 00000000000000000000000000000000 P 14944af41599467eb39a5490a6d3d67d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14944af41599467eb39a5490a6d3d67d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 41739 } }
I20260925 07:25:24.662505 8725 raft_consensus.cc:374] T 00000000000000000000000000000000 P 14944af41599467eb39a5490a6d3d67d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:25:24.662565 8725 raft_consensus.cc:729] T 00000000000000000000000000000000 P 14944af41599467eb39a5490a6d3d67d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14944af41599467eb39a5490a6d3d67d, State: Initialized, Role: FOLLOWER
I20260925 07:25:24.662689 8725 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14944af41599467eb39a5490a6d3d67d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14944af41599467eb39a5490a6d3d67d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 41739 } }
I20260925 07:25:24.662758 8725 raft_consensus.cc:388] T 00000000000000000000000000000000 P 14944af41599467eb39a5490a6d3d67d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 07:25:24.662801 8725 raft_consensus.cc:482] T 00000000000000000000000000000000 P 14944af41599467eb39a5490a6d3d67d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 07:25:24.662846 8725 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 14944af41599467eb39a5490a6d3d67d [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:25:24.662979 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 8657
I20260925 07:25:24.663075 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790321110323796-5753-0/minicluster-data/master-0/wal/instance
I20260925 07:25:24.663219 5753 external_mini_cluster.cc:1428] Setting key 8291ba9bbb6c2dd8bec6173aba9c3fd2
I20260925 07:25:24.663795 8725 raft_consensus.cc:504] T 00000000000000000000000000000000 P 14944af41599467eb39a5490a6d3d67d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14944af41599467eb39a5490a6d3d67d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 41739 } }
I20260925 07:25:24.663911 8725 leader_election.cc:302] T 00000000000000000000000000000000 P 14944af41599467eb39a5490a6d3d67d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 14944af41599467eb39a5490a6d3d67d; no voters:
I20260925 07:25:24.664048 8725 leader_election.cc:288] T 00000000000000000000000000000000 P 14944af41599467eb39a5490a6d3d67d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 07:25:24.664098 8730 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 14944af41599467eb39a5490a6d3d67d [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:25:24.664247 8730 raft_consensus.cc:686] T 00000000000000000000000000000000 P 14944af41599467eb39a5490a6d3d67d [term 1 LEADER]: Becoming Leader. State: Replica: 14944af41599467eb39a5490a6d3d67d, State: Running, Role: LEADER
I20260925 07:25:24.664273 8725 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14944af41599467eb39a5490a6d3d67d [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:25:24.665210 8730 consensus_queue.cc:237] T 00000000000000000000000000000000 P 14944af41599467eb39a5490a6d3d67d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14944af41599467eb39a5490a6d3d67d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 41739 } }
I20260925 07:25:24.665684 8732 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14944af41599467eb39a5490a6d3d67d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "14944af41599467eb39a5490a6d3d67d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14944af41599467eb39a5490a6d3d67d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 41739 } } }
I20260925 07:25:24.665910 8732 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14944af41599467eb39a5490a6d3d67d [sys.catalog]: This master's current role is: LEADER
W20260925 07:25:24.666039 8744 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 14944af41599467eb39a5490a6d3d67d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 07:25:24.666085 8744 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 07:25:24.666194 8731 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14944af41599467eb39a5490a6d3d67d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 14944af41599467eb39a5490a6d3d67d. Latest consensus state: current_term: 1 leader_uuid: "14944af41599467eb39a5490a6d3d67d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14944af41599467eb39a5490a6d3d67d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 41739 } } }
I20260925 07:25:24.666258 8731 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14944af41599467eb39a5490a6d3d67d [sys.catalog]: This master's current role is: LEADER
I20260925 07:25:24.669216 8746 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 07:25:24.669395 8746 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 25 07:25:24 dist-test-slave-1drq krb5kdc[8644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321124, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:25:24.671247 8746 catalog_manager.cc:1347] Generated new cluster ID: 610305a852754892882793da64b56bea
I20260925 07:25:24.671294 8746 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 07:25:24.673938 8729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:24.663643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34102 (local address 127.5.158.126:41739)
0925 07:25:24.663808 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:24.663834 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:24.663865 (+ 31us) server_negotiation.cc:407] Connection header received
0925 07:25:24.663969 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:24.663976 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:24.664042 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:24.664890 (+ 848us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:24.664952 (+ 62us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:24.666257 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:24.667157 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:24.667741 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:24.667850 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:24.670146 (+ 2296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:24.670150 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:24.670150 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:24.670160 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:24.672142 (+ 1982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:24.672553 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:24.672556 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:24.672556 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:24.672692 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:24.672948 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:24.672949 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:24.672949 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:24.673330 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:24.673547 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:24.673634 (+ 87us) server_negotiation.cc:299] Negotiation successful
0925 07:25:24.673775 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":82,"threads_started":1}
I20260925 07:25:24.677870 5753 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 07:25:24.686389 8746 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 07:25:24.687078 8746 catalog_manager.cc:1504] Loading token signing keys...
I20260925 07:25:24.692905 8746 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 14944af41599467eb39a5490a6d3d67d: Generated new TSK 0
I20260925 07:25:24.693117 8746 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 07:25:24.694952 8729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:24.691592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34112 (local address 127.5.158.126:41739)
0925 07:25:24.691623 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:24.691624 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:24.691723 (+ 99us) server_negotiation.cc:407] Connection header received
0925 07:25:24.691801 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:24.691802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:24.691811 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:24.691835 (+ 24us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:24.692214 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:24.692902 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:24.693660 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:24.693957 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:24.694033 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:24.694560 (+ 527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:24.694561 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:24.694562 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:25:24.694739 (+ 177us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:24.694754 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:25:24.694760 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:24.694816 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:24.694865 (+ 49us) server_negotiation.cc:299] Negotiation successful
0925 07:25:24.694888 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260925 07:25:24.697217 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 8657
2026-09-25T07:25:24Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1196 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 07:25:24 dist-test-slave-1drq krb5kdc[8758](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 07:25:24 dist-test-slave-1drq krb5kdc[8758](info): set up 2 sockets
Sep 25 07:25:24 dist-test-slave-1drq krb5kdc[8758](info): commencing operation
krb5kdc: starting...
W20260925 07:25:25.636463 5753 mini_kdc.cc:121] Time spent starting KDC: real 0.922s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Sep 25 07:25:25 dist-test-slave-1drq krb5kdc[8758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321125, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T07:25:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T07:25:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
I20260925 07:25:25.737295 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:39125
--webserver_interface=127.5.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:45675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.158.126:39125
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:25:25.809810 8771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:25:25.810017 8771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:25:25.810050 8771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:25:25.811158 8771 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:25:25.811223 8771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:25:25.811247 8771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:25:25.811277 8771 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:25:25.811311 8771 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:25:25.812572 8771 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:45675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.158.126:39125
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:39125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8771
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:25:25.812800 8771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:25:25.812984 8771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:25:25.817152 8779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:25.817126 8776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:25.817148 8771 server_base.cc:1034] running on GCE node
W20260925 07:25:25.817133 8777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:25.817642 8771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:25:25.817914 8771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:25:25.819047 8771 hybrid_clock.cc:648] HybridClock initialized: now 1790321125819039 us; error 26 us; skew 500 ppm
Sep 25 07:25:25 dist-test-slave-1drq krb5kdc[8758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321125, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:25:25.825223 8771 init.cc:377] Logged in from keytab as kudu/127.5.158.126@KRBTEST.COM (short username kudu)
I20260925 07:25:25.825587 8771 webserver.cc:460] Webserver started at http://127.5.158.126:40109/ using document root <none> and password file <none>
I20260925 07:25:25.825789 8771 fs_manager.cc:359] Metadata directory not provided
I20260925 07:25:25.825846 8771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:25:25.825963 8771 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:25:25.826833 8771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/master-0/data/instance:
uuid: "f44ac7e387bb4b2f9a0909e24a56fe90"
format_stamp: "Formatted at 2026-09-25 07:25:25 on dist-test-slave-1drq"
server_key: "0c6f581f66b127f74b4470095f8828fd"
server_key_iv: "8bc3c694affd0928a68b1e56aa89078f"
server_key_version: "encryptionkey@0"
I20260925 07:25:25.827150 8771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/master-0/wal/instance:
uuid: "f44ac7e387bb4b2f9a0909e24a56fe90"
format_stamp: "Formatted at 2026-09-25 07:25:25 on dist-test-slave-1drq"
server_key: "0c6f581f66b127f74b4470095f8828fd"
server_key_iv: "8bc3c694affd0928a68b1e56aa89078f"
server_key_version: "encryptionkey@0"
I20260925 07:25:25.828392 8771 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 07:25:25.829067 8786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:25.829282 8771 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:25:25.829375 8771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "f44ac7e387bb4b2f9a0909e24a56fe90"
format_stamp: "Formatted at 2026-09-25 07:25:25 on dist-test-slave-1drq"
server_key: "0c6f581f66b127f74b4470095f8828fd"
server_key_iv: "8bc3c694affd0928a68b1e56aa89078f"
server_key_version: "encryptionkey@0"
I20260925 07:25:25.829453 8771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:25:25.838932 8771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:25:25.840812 8771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:25:25.840947 8771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:25:25.844300 8771 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.126:39125
I20260925 07:25:25.844326 8838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.126:39125 every 8 connection(s)
I20260925 07:25:25.844617 8771 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:25:25.845562 8839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:25:25.847411 8839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f44ac7e387bb4b2f9a0909e24a56fe90: Bootstrap starting.
I20260925 07:25:25.847932 8839 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f44ac7e387bb4b2f9a0909e24a56fe90: Neither blocks nor log segments found. Creating new log.
I20260925 07:25:25.848213 8839 log.cc:824] T 00000000000000000000000000000000 P f44ac7e387bb4b2f9a0909e24a56fe90: Log is configured to *not* fsync() on all Append() calls
I20260925 07:25:25.848814 8839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f44ac7e387bb4b2f9a0909e24a56fe90: No bootstrap required, opened a new log
I20260925 07:25:25.850142 8839 raft_consensus.cc:348] T 00000000000000000000000000000000 P f44ac7e387bb4b2f9a0909e24a56fe90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f44ac7e387bb4b2f9a0909e24a56fe90" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 39125 } }
I20260925 07:25:25.850265 8839 raft_consensus.cc:374] T 00000000000000000000000000000000 P f44ac7e387bb4b2f9a0909e24a56fe90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:25:25.850309 8839 raft_consensus.cc:729] T 00000000000000000000000000000000 P f44ac7e387bb4b2f9a0909e24a56fe90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f44ac7e387bb4b2f9a0909e24a56fe90, State: Initialized, Role: FOLLOWER
I20260925 07:25:25.850441 8839 consensus_queue.cc:260] T 00000000000000000000000000000000 P f44ac7e387bb4b2f9a0909e24a56fe90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f44ac7e387bb4b2f9a0909e24a56fe90" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 39125 } }
I20260925 07:25:25.850497 8839 raft_consensus.cc:388] T 00000000000000000000000000000000 P f44ac7e387bb4b2f9a0909e24a56fe90 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 07:25:25.850514 8839 raft_consensus.cc:482] T 00000000000000000000000000000000 P f44ac7e387bb4b2f9a0909e24a56fe90 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 07:25:25.850576 8839 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f44ac7e387bb4b2f9a0909e24a56fe90 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:25:25.850875 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 8771
I20260925 07:25:25.850975 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/master-0/wal/instance
I20260925 07:25:25.851135 5753 external_mini_cluster.cc:1428] Setting key 264572354c9b0ddd616e5a2375a202d7
I20260925 07:25:25.851372 8839 raft_consensus.cc:504] T 00000000000000000000000000000000 P f44ac7e387bb4b2f9a0909e24a56fe90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f44ac7e387bb4b2f9a0909e24a56fe90" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 39125 } }
I20260925 07:25:25.851513 8839 leader_election.cc:302] T 00000000000000000000000000000000 P f44ac7e387bb4b2f9a0909e24a56fe90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f44ac7e387bb4b2f9a0909e24a56fe90; no voters:
I20260925 07:25:25.851653 8839 leader_election.cc:288] T 00000000000000000000000000000000 P f44ac7e387bb4b2f9a0909e24a56fe90 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 07:25:25.851686 8843 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f44ac7e387bb4b2f9a0909e24a56fe90 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:25:25.851861 8839 sys_catalog.cc:564] T 00000000000000000000000000000000 P f44ac7e387bb4b2f9a0909e24a56fe90 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:25:25.852265 8843 raft_consensus.cc:686] T 00000000000000000000000000000000 P f44ac7e387bb4b2f9a0909e24a56fe90 [term 1 LEADER]: Becoming Leader. State: Replica: f44ac7e387bb4b2f9a0909e24a56fe90, State: Running, Role: LEADER
I20260925 07:25:25.852355 8843 consensus_queue.cc:237] T 00000000000000000000000000000000 P f44ac7e387bb4b2f9a0909e24a56fe90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f44ac7e387bb4b2f9a0909e24a56fe90" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 39125 } }
I20260925 07:25:25.852660 8843 sys_catalog.cc:455] T 00000000000000000000000000000000 P f44ac7e387bb4b2f9a0909e24a56fe90 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f44ac7e387bb4b2f9a0909e24a56fe90. Latest consensus state: current_term: 1 leader_uuid: "f44ac7e387bb4b2f9a0909e24a56fe90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f44ac7e387bb4b2f9a0909e24a56fe90" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 39125 } } }
I20260925 07:25:25.852689 8845 sys_catalog.cc:455] T 00000000000000000000000000000000 P f44ac7e387bb4b2f9a0909e24a56fe90 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f44ac7e387bb4b2f9a0909e24a56fe90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f44ac7e387bb4b2f9a0909e24a56fe90" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 39125 } } }
I20260925 07:25:25.852876 8845 sys_catalog.cc:458] T 00000000000000000000000000000000 P f44ac7e387bb4b2f9a0909e24a56fe90 [sys.catalog]: This master's current role is: LEADER
I20260925 07:25:25.853075 8843 sys_catalog.cc:458] T 00000000000000000000000000000000 P f44ac7e387bb4b2f9a0909e24a56fe90 [sys.catalog]: This master's current role is: LEADER
I20260925 07:25:25.853103 8853 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 07:25:25.853615 8853 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 07:25:25.854883 8861 mvcc.cc:205] Tried to move back new op lower bound from 7333155331499876352 to 7333155331491573760. Current Snapshot: MvccSnapshot[applied={T|T < 7333155331499876352}]
I20260925 07:25:25.857501 8853 catalog_manager.cc:1347] Generated new cluster ID: 4c7357605be14bfd8f94f180998551b1
I20260925 07:25:25.857582 8853 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 07:25:25 dist-test-slave-1drq krb5kdc[8758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321125, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:25:25.861006 8844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:25.851536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44244 (local address 127.5.158.126:39125)
0925 07:25:25.851815 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:25.851821 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:25.851842 (+ 21us) server_negotiation.cc:407] Connection header received
0925 07:25:25.851918 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:25.851923 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:25.851956 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:25.852095 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:25.852825 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:25.854189 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:25.855175 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:25.855566 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:25.855638 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:25.858115 (+ 2477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:25.858119 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:25.858119 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:25.858129 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:25.859588 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:25.859924 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:25.859927 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:25.859927 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:25.860013 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:25.860215 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:25.860216 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:25.860217 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:25.860439 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:25.860612 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:25.860709 (+ 97us) server_negotiation.cc:299] Negotiation successful
0925 07:25:25.860838 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":96,"threads_started":1}
I20260925 07:25:25.870563 8853 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 07:25:25.871160 8853 catalog_manager.cc:1504] Loading token signing keys...
I20260925 07:25:25.885412 8853 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f44ac7e387bb4b2f9a0909e24a56fe90: Generated new TSK 0
I20260925 07:25:25.885565 8853 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.158.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab.
Entry for principal kudu/127.5.158.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab.
I20260925 07:25:25.911252 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.65:0
--local_ip_for_outbound_sockets=127.5.158.65
--webserver_interface=127.5.158.65
--webserver_port=0
--tserver_master_addrs=127.5.158.126:39125
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab
--principal=kudu/127.5.158.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:45675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:25:25.985818 8865 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:25:25.985981 8865 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:25:25.986001 8865 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:25:25.986954 8865 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:25:25.986986 8865 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:25:25.987010 8865 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.65
I20260925 07:25:25.988313 8865 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:45675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab
--principal=kudu/127.5.158.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.158.65
--webserver_port=0
--tserver_master_addrs=127.5.158.126:39125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8865
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.65
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:25:25.988487 8865 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:25:25.988646 8865 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:25:25.992969 8870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:25.993047 8871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:25.993021 8865 server_base.cc:1034] running on GCE node
W20260925 07:25:25.992974 8873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:25.993389 8865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:25:25.993656 8865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:25:25.994793 8865 hybrid_clock.cc:648] HybridClock initialized: now 1790321125994773 us; error 39 us; skew 500 ppm
Sep 25 07:25:25 dist-test-slave-1drq krb5kdc[8758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321125, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:25:25.996980 8865 init.cc:377] Logged in from keytab as kudu/127.5.158.65@KRBTEST.COM (short username kudu)
I20260925 07:25:25.997418 8865 webserver.cc:460] Webserver started at http://127.5.158.65:40609/ using document root <none> and password file <none>
I20260925 07:25:25.997623 8865 fs_manager.cc:359] Metadata directory not provided
I20260925 07:25:25.997666 8865 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:25:25.997802 8865 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:25:25.998728 8865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-0/data/instance:
uuid: "a117d6f9c1ee4af08500dec6cde8dc8d"
format_stamp: "Formatted at 2026-09-25 07:25:25 on dist-test-slave-1drq"
server_key: "fec7d929806a6b02649f029dd05e348b"
server_key_iv: "ad14239c762fcb2477f021ab817f57b3"
server_key_version: "encryptionkey@0"
I20260925 07:25:25.999053 8865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-0/wal/instance:
uuid: "a117d6f9c1ee4af08500dec6cde8dc8d"
format_stamp: "Formatted at 2026-09-25 07:25:25 on dist-test-slave-1drq"
server_key: "fec7d929806a6b02649f029dd05e348b"
server_key_iv: "ad14239c762fcb2477f021ab817f57b3"
server_key_version: "encryptionkey@0"
I20260925 07:25:26.000242 8865 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 07:25:26.000849 8880 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:26.001008 8865 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 07:25:26.001124 8865 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-0/wal
uuid: "a117d6f9c1ee4af08500dec6cde8dc8d"
format_stamp: "Formatted at 2026-09-25 07:25:25 on dist-test-slave-1drq"
server_key: "fec7d929806a6b02649f029dd05e348b"
server_key_iv: "ad14239c762fcb2477f021ab817f57b3"
server_key_version: "encryptionkey@0"
I20260925 07:25:26.001202 8865 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:25:26.007895 8865 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:25:26.009523 8865 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:25:26.009665 8865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:25:26.009852 8865 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:25:26.010109 8865 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:25:26.010165 8865 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:26.010202 8865 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:25:26.010229 8865 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:26.015468 8865 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.65:35257
I20260925 07:25:26.015547 8993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.65:35257 every 8 connection(s)
I20260925 07:25:26.015784 8865 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
Sep 25 07:25:26 dist-test-slave-1drq krb5kdc[8758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321125, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.65@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:25:26.023505 8844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:26.016358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:53093 (local address 127.5.158.126:39125)
0925 07:25:26.016413 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:26.016415 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:26.016593 (+ 178us) server_negotiation.cc:407] Connection header received
0925 07:25:26.016968 (+ 375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:26.016969 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:26.016977 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:26.017024 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:26.017900 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:26.018379 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:26.019214 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:26.019378 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:26.019423 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:26.021202 (+ 1779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:26.021203 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:26.021204 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:26.021209 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:26.022506 (+ 1297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:26.022832 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:26.022833 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:26.022833 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:26.022877 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:26.023059 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:26.023059 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:26.023060 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:26.023145 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:26.023189 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:26.023387 (+ 198us) server_negotiation.cc:299] Negotiation successful
0925 07:25:26.023447 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260925 07:25:26.023643 8996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:26.016341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:39125 (local address 127.5.158.65:53093)
0925 07:25:26.016480 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:26.016514 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:26.016898 (+ 384us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:26.017063 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:26.017066 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:26.017661 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:26.017827 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:26.017831 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:26.018463 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:26.018464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:26.019144 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:26.019146 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:26.019440 (+ 294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:26.019440 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:26.019524 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:26.019870 (+ 346us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:26.019877 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:26.021044 (+ 1167us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:26.022587 (+ 1543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:26.022592 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:26.022593 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:26.022776 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:26.022929 (+ 153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:26.022930 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:26.022930 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:26.023012 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:26.023196 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:26.023200 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:26.023292 (+ 92us) client_negotiation.cc:769] Sending connection context
0925 07:25:26.023363 (+ 71us) client_negotiation.cc:240] Negotiation successful
0925 07:25:26.023460 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":82,"threads_started":1}
I20260925 07:25:26.024076 8994 heartbeater.cc:347] Connected to a master server at 127.5.158.126:39125
I20260925 07:25:26.024169 8994 heartbeater.cc:464] Registering TS with master...
I20260925 07:25:26.024327 8994 heartbeater.cc:511] Master 127.5.158.126:39125 requested a full tablet report, sending...
I20260925 07:25:26.024756 8803 ts_manager.cc:194] Registered new tserver with Master: a117d6f9c1ee4af08500dec6cde8dc8d (127.5.158.65:35257)
I20260925 07:25:26.025058 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 8865
I20260925 07:25:26.025177 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-0/wal/instance
I20260925 07:25:26.025352 5753 external_mini_cluster.cc:1428] Setting key d4edf303aa4041284eb528b7fa741ea1
I20260925 07:25:26.025468 8803 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.65@KRBTEST.COM'} at 127.5.158.65:53093
WARNING: no policy specified for kudu/127.5.158.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab.
Entry for principal kudu/127.5.158.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab.
I20260925 07:25:26.045379 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.66:0
--local_ip_for_outbound_sockets=127.5.158.66
--webserver_interface=127.5.158.66
--webserver_port=0
--tserver_master_addrs=127.5.158.126:39125
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab
--principal=kudu/127.5.158.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:45675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:25:26.115715 8999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:25:26.115893 8999 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:25:26.115928 8999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:25:26.116979 8999 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:25:26.117036 8999 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:25:26.117077 8999 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.66
I20260925 07:25:26.118486 8999 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:45675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab
--principal=kudu/127.5.158.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.158.66
--webserver_port=0
--tserver_master_addrs=127.5.158.126:39125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8999
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.66
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:25:26.118718 8999 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:25:26.118903 8999 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 07:25:26.123168 8999 server_base.cc:1034] running on GCE node
W20260925 07:25:26.123147 9005 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:26.123152 9007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:26.123170 9004 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:26.123545 8999 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:25:26.123713 8999 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:25:26.124845 8999 hybrid_clock.cc:648] HybridClock initialized: now 1790321126124830 us; error 32 us; skew 500 ppm
Sep 25 07:25:26 dist-test-slave-1drq krb5kdc[8758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321126, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:25:26.126941 8999 init.cc:377] Logged in from keytab as kudu/127.5.158.66@KRBTEST.COM (short username kudu)
I20260925 07:25:26.127360 8999 webserver.cc:460] Webserver started at http://127.5.158.66:34967/ using document root <none> and password file <none>
I20260925 07:25:26.127579 8999 fs_manager.cc:359] Metadata directory not provided
I20260925 07:25:26.127651 8999 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:25:26.127799 8999 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:25:26.128701 8999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-1/data/instance:
uuid: "0964084d5f8d40a493728015500fa632"
format_stamp: "Formatted at 2026-09-25 07:25:26 on dist-test-slave-1drq"
server_key: "0430e5d01266c7d61227688f9aeb0aeb"
server_key_iv: "9ab523080ebd9bb2fff72d0bf9fa3f46"
server_key_version: "encryptionkey@0"
I20260925 07:25:26.129009 8999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-1/wal/instance:
uuid: "0964084d5f8d40a493728015500fa632"
format_stamp: "Formatted at 2026-09-25 07:25:26 on dist-test-slave-1drq"
server_key: "0430e5d01266c7d61227688f9aeb0aeb"
server_key_iv: "9ab523080ebd9bb2fff72d0bf9fa3f46"
server_key_version: "encryptionkey@0"
I20260925 07:25:26.130107 8999 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 07:25:26.130743 9014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:26.130893 8999 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 07:25:26.130977 8999 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-1/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-1/wal
uuid: "0964084d5f8d40a493728015500fa632"
format_stamp: "Formatted at 2026-09-25 07:25:26 on dist-test-slave-1drq"
server_key: "0430e5d01266c7d61227688f9aeb0aeb"
server_key_iv: "9ab523080ebd9bb2fff72d0bf9fa3f46"
server_key_version: "encryptionkey@0"
I20260925 07:25:26.131052 8999 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:25:26.142642 8999 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:25:26.144780 8999 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:25:26.144912 8999 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:25:26.145153 8999 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:25:26.145442 8999 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:25:26.145483 8999 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:26.145516 8999 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:25:26.145537 8999 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:26.150810 8999 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.66:42973
I20260925 07:25:26.150848 9127 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.66:42973 every 8 connection(s)
I20260925 07:25:26.151084 8999 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
Sep 25 07:25:26 dist-test-slave-1drq krb5kdc[8758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321126, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.66@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:25:26.158582 8844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:26.151680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:56817 (local address 127.5.158.126:39125)
0925 07:25:26.151742 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:26.151744 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:26.151870 (+ 126us) server_negotiation.cc:407] Connection header received
0925 07:25:26.152238 (+ 368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:26.152239 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:26.152248 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:26.152293 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:26.153136 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:26.153554 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:26.154385 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:26.154547 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:26.154593 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:26.156328 (+ 1735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:26.156329 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:26.156330 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:26.156335 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:26.157536 (+ 1201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:26.157868 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:26.157868 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:26.157869 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:26.157914 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:26.158105 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:26.158105 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:26.158105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:26.158196 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:26.158243 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:26.158466 (+ 223us) server_negotiation.cc:299] Negotiation successful
0925 07:25:26.158523 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 07:25:26.158753 9130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:26.151635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:39125 (local address 127.5.158.66:56817)
0925 07:25:26.151773 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:26.151800 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:26.152152 (+ 352us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:26.152333 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:26.152336 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:26.152921 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:26.153067 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:26.153070 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:26.153619 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:26.153619 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:26.154324 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:26.154326 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:26.154608 (+ 282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:26.154608 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:26.154696 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:26.155086 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:26.155093 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:26.156259 (+ 1166us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:26.157602 (+ 1343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:26.157607 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:26.157608 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:26.157804 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:26.157969 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:26.157969 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:26.157969 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:26.158053 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:26.158248 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:26.158252 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:26.158345 (+ 93us) client_negotiation.cc:769] Sending connection context
0925 07:25:26.158430 (+ 85us) client_negotiation.cc:240] Negotiation successful
0925 07:25:26.158533 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":77,"threads_started":1}
I20260925 07:25:26.158927 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 8999
I20260925 07:25:26.159034 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-1/wal/instance
I20260925 07:25:26.159190 9128 heartbeater.cc:347] Connected to a master server at 127.5.158.126:39125
I20260925 07:25:26.159214 5753 external_mini_cluster.cc:1428] Setting key 2e1acffa384cedfc380d42a5b0c120c1
I20260925 07:25:26.159281 9128 heartbeater.cc:464] Registering TS with master...
I20260925 07:25:26.159441 9128 heartbeater.cc:511] Master 127.5.158.126:39125 requested a full tablet report, sending...
I20260925 07:25:26.159785 8802 ts_manager.cc:194] Registered new tserver with Master: 0964084d5f8d40a493728015500fa632 (127.5.158.66:42973)
I20260925 07:25:26.160153 8802 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.66@KRBTEST.COM'} at 127.5.158.66:56817
WARNING: no policy specified for kudu/127.5.158.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab.
Entry for principal kudu/127.5.158.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab.
I20260925 07:25:26.179567 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.67:0
--local_ip_for_outbound_sockets=127.5.158.67
--webserver_interface=127.5.158.67
--webserver_port=0
--tserver_master_addrs=127.5.158.126:39125
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab
--principal=kudu/127.5.158.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:45675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:25:26.249632 9133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:25:26.249782 9133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:25:26.249801 9133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:25:26.250725 9133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:25:26.250752 9133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:25:26.250774 9133 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.67
I20260925 07:25:26.252017 9133 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:45675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab
--principal=kudu/127.5.158.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.158.67
--webserver_port=0
--tserver_master_addrs=127.5.158.126:39125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9133
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.67
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:25:26.252242 9133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:25:26.252386 9133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:25:26.256594 9141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:26.256630 9139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:26.256717 9133 server_base.cc:1034] running on GCE node
W20260925 07:25:26.256664 9138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:26.256922 9133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:25:26.257134 9133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:25:26.258273 9133 hybrid_clock.cc:648] HybridClock initialized: now 1790321126258261 us; error 30 us; skew 500 ppm
Sep 25 07:25:26 dist-test-slave-1drq krb5kdc[8758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321126, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:25:26.260401 9133 init.cc:377] Logged in from keytab as kudu/127.5.158.67@KRBTEST.COM (short username kudu)
I20260925 07:25:26.260780 9133 webserver.cc:460] Webserver started at http://127.5.158.67:43901/ using document root <none> and password file <none>
I20260925 07:25:26.260943 9133 fs_manager.cc:359] Metadata directory not provided
I20260925 07:25:26.260982 9133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:25:26.261062 9133 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:25:26.261915 9133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-2/data/instance:
uuid: "089a5934d8f8405aa47e484c977b970c"
format_stamp: "Formatted at 2026-09-25 07:25:26 on dist-test-slave-1drq"
server_key: "21da7cc8fb66132539eb27f5f7cef5bb"
server_key_iv: "be1a40d430de9bdd1c720e0b7c6f624c"
server_key_version: "encryptionkey@0"
I20260925 07:25:26.262176 9133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-2/wal/instance:
uuid: "089a5934d8f8405aa47e484c977b970c"
format_stamp: "Formatted at 2026-09-25 07:25:26 on dist-test-slave-1drq"
server_key: "21da7cc8fb66132539eb27f5f7cef5bb"
server_key_iv: "be1a40d430de9bdd1c720e0b7c6f624c"
server_key_version: "encryptionkey@0"
I20260925 07:25:26.263229 9133 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 07:25:26.263772 9148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:26.263917 9133 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 07:25:26.263999 9133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-2/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-2/wal
uuid: "089a5934d8f8405aa47e484c977b970c"
format_stamp: "Formatted at 2026-09-25 07:25:26 on dist-test-slave-1drq"
server_key: "21da7cc8fb66132539eb27f5f7cef5bb"
server_key_iv: "be1a40d430de9bdd1c720e0b7c6f624c"
server_key_version: "encryptionkey@0"
I20260925 07:25:26.264074 9133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:25:26.274806 9133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:25:26.276268 9133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:25:26.276401 9133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:25:26.276607 9133 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:25:26.276863 9133 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:25:26.276914 9133 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:26.276950 9133 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:25:26.276978 9133 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:26.282295 9133 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.67:38713
I20260925 07:25:26.282348 9261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.67:38713 every 8 connection(s)
I20260925 07:25:26.282584 9133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
I20260925 07:25:26.283084 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 9133
I20260925 07:25:26.283185 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790321110323796-5753-0/minicluster-data/ts-2/wal/instance
I20260925 07:25:26.283372 5753 external_mini_cluster.cc:1428] Setting key 0bf056e2d14c390f13c10ddfdde4df91
Sep 25 07:25:26 dist-test-slave-1drq krb5kdc[8758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321126, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.67@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:25:26.290587 8844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:26.283170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:33363 (local address 127.5.158.126:39125)
0925 07:25:26.283217 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:26.283219 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:26.283436 (+ 217us) server_negotiation.cc:407] Connection header received
0925 07:25:26.283926 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:26.283927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:26.283937 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:26.283987 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:26.284838 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:26.285409 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:26.286241 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:26.286400 (+ 159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:26.286443 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:26.288275 (+ 1832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:26.288276 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:26.288276 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:26.288282 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:26.289551 (+ 1269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:26.289884 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:26.289885 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:26.289885 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:26.289933 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:26.290124 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:26.290124 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:26.290124 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:26.290228 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:26.290275 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:26.290475 (+ 200us) server_negotiation.cc:299] Negotiation successful
0925 07:25:26.290525 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260925 07:25:26.290710 9264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:26.283129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:39125 (local address 127.5.158.67:33363)
0925 07:25:26.283304 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:26.283345 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:26.283824 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:26.284034 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:26.284037 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:26.284601 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:26.284765 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:26.284769 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:26.285486 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:26.285487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:26.286173 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:26.286175 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:26.286459 (+ 284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:26.286460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:26.286540 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:26.286965 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:26.286971 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:26.288198 (+ 1227us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:26.289622 (+ 1424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:26.289627 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:26.289628 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:26.289823 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:26.289987 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:26.289988 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:26.289988 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:26.290072 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:26.290278 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:26.290282 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:26.290382 (+ 100us) client_negotiation.cc:769] Sending connection context
0925 07:25:26.290451 (+ 69us) client_negotiation.cc:240] Negotiation successful
0925 07:25:26.290542 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":71,"threads_started":1}
I20260925 07:25:26.291115 9262 heartbeater.cc:347] Connected to a master server at 127.5.158.126:39125
I20260925 07:25:26.291182 9262 heartbeater.cc:464] Registering TS with master...
I20260925 07:25:26.291286 9262 heartbeater.cc:511] Master 127.5.158.126:39125 requested a full tablet report, sending...
I20260925 07:25:26.291569 8802 ts_manager.cc:194] Registered new tserver with Master: 089a5934d8f8405aa47e484c977b970c (127.5.158.67:38713)
I20260925 07:25:26.291867 8802 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.67@KRBTEST.COM'} at 127.5.158.67:33363
I20260925 07:25:26.295023 5753 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 25 07:25:26 dist-test-slave-1drq krb5kdc[8758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321126, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 25 07:25:26 dist-test-slave-1drq krb5kdc[8758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321126, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:25:26.314599 8844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:26.308753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44260 (local address 127.5.158.126:39125)
0925 07:25:26.308817 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:26.308819 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:26.308913 (+ 94us) server_negotiation.cc:407] Connection header received
0925 07:25:26.308982 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:26.308983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:26.308994 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:26.309052 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:26.309588 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:26.309985 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:26.310641 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:26.310802 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:26.310876 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:26.312695 (+ 1819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:26.312696 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:26.312696 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:26.312701 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:26.313780 (+ 1079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:26.314045 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:26.314046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:26.314046 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:26.314113 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:26.314254 (+ 141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:26.314255 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:26.314255 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:26.314359 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:26.314421 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:26.314494 (+ 73us) server_negotiation.cc:299] Negotiation successful
0925 07:25:26.314543 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
W20260925 07:25:26.314790 8802 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:44260
I20260925 07:25:26.315186 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 8865
I20260925 07:25:26.319480 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 8999
I20260925 07:25:26.323432 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 9133
I20260925 07:25:26.327236 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 8771
2026-09-25T07:25:26Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1633 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 07:25:26 dist-test-slave-1drq krb5kdc[9274](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 07:25:26 dist-test-slave-1drq krb5kdc[9274](info): set up 2 sockets
Sep 25 07:25:26 dist-test-slave-1drq krb5kdc[9274](info): commencing operation
krb5kdc: starting...
W20260925 07:25:27.264376 5753 mini_kdc.cc:121] Time spent starting KDC: real 0.917s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Sep 25 07:25:27 dist-test-slave-1drq krb5kdc[9274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T07:25:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T07:25:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
I20260925 07:25:27.363790 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:33799
--webserver_interface=127.5.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:45383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.158.126:33799
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:25:27.434973 9287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:25:27.435178 9287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:25:27.435230 9287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:25:27.436326 9287 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:25:27.436381 9287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:25:27.436403 9287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:25:27.436421 9287 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:25:27.436448 9287 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:25:27.437844 9287 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:45383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.158.126:33799
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:33799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9287
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:25:27.438100 9287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:25:27.438304 9287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:25:27.442552 9293 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:27.442579 9292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:27.442617 9287 server_base.cc:1034] running on GCE node
W20260925 07:25:27.442552 9295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:27.442914 9287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:25:27.443156 9287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:25:27.444288 9287 hybrid_clock.cc:648] HybridClock initialized: now 1790321127444276 us; error 24 us; skew 500 ppm
Sep 25 07:25:27 dist-test-slave-1drq krb5kdc[9274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321127, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:25:27.446321 9287 init.cc:377] Logged in from keytab as kudu/127.5.158.126@KRBTEST.COM (short username kudu)
I20260925 07:25:27.446683 9287 webserver.cc:460] Webserver started at http://127.5.158.126:45737/ using document root <none> and password file <none>
I20260925 07:25:27.446846 9287 fs_manager.cc:359] Metadata directory not provided
I20260925 07:25:27.446878 9287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:25:27.446954 9287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:25:27.447834 9287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/master-0/data/instance:
uuid: "14ab1951295941e9b8f4c89200325711"
format_stamp: "Formatted at 2026-09-25 07:25:27 on dist-test-slave-1drq"
server_key: "74da8f25b3613e0c6ee411554966ccae"
server_key_iv: "734bc53cc74782245293933baf260433"
server_key_version: "encryptionkey@0"
I20260925 07:25:27.448160 9287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/master-0/wal/instance:
uuid: "14ab1951295941e9b8f4c89200325711"
format_stamp: "Formatted at 2026-09-25 07:25:27 on dist-test-slave-1drq"
server_key: "74da8f25b3613e0c6ee411554966ccae"
server_key_iv: "734bc53cc74782245293933baf260433"
server_key_version: "encryptionkey@0"
I20260925 07:25:27.449407 9287 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 07:25:27.450001 9302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:27.450132 9287 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 07:25:27.450229 9287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "14ab1951295941e9b8f4c89200325711"
format_stamp: "Formatted at 2026-09-25 07:25:27 on dist-test-slave-1drq"
server_key: "74da8f25b3613e0c6ee411554966ccae"
server_key_iv: "734bc53cc74782245293933baf260433"
server_key_version: "encryptionkey@0"
I20260925 07:25:27.450299 9287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:25:27.472823 9287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:25:27.474386 9287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:25:27.474530 9287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:25:27.478189 9287 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.126:33799
I20260925 07:25:27.478227 9354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.126:33799 every 8 connection(s)
I20260925 07:25:27.478703 9287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:25:27.479233 9355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:25:27.481029 9355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14ab1951295941e9b8f4c89200325711: Bootstrap starting.
I20260925 07:25:27.481518 9355 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 14ab1951295941e9b8f4c89200325711: Neither blocks nor log segments found. Creating new log.
I20260925 07:25:27.481727 9355 log.cc:824] T 00000000000000000000000000000000 P 14ab1951295941e9b8f4c89200325711: Log is configured to *not* fsync() on all Append() calls
I20260925 07:25:27.482275 9355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14ab1951295941e9b8f4c89200325711: No bootstrap required, opened a new log
I20260925 07:25:27.483564 9355 raft_consensus.cc:348] T 00000000000000000000000000000000 P 14ab1951295941e9b8f4c89200325711 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14ab1951295941e9b8f4c89200325711" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 33799 } }
I20260925 07:25:27.483673 9355 raft_consensus.cc:374] T 00000000000000000000000000000000 P 14ab1951295941e9b8f4c89200325711 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:25:27.483692 9355 raft_consensus.cc:729] T 00000000000000000000000000000000 P 14ab1951295941e9b8f4c89200325711 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14ab1951295941e9b8f4c89200325711, State: Initialized, Role: FOLLOWER
I20260925 07:25:27.483812 9355 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14ab1951295941e9b8f4c89200325711 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14ab1951295941e9b8f4c89200325711" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 33799 } }
I20260925 07:25:27.483889 9355 raft_consensus.cc:388] T 00000000000000000000000000000000 P 14ab1951295941e9b8f4c89200325711 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 07:25:27.483911 9355 raft_consensus.cc:482] T 00000000000000000000000000000000 P 14ab1951295941e9b8f4c89200325711 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 07:25:27.483934 9355 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 14ab1951295941e9b8f4c89200325711 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:25:27.484638 9355 raft_consensus.cc:504] T 00000000000000000000000000000000 P 14ab1951295941e9b8f4c89200325711 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14ab1951295941e9b8f4c89200325711" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 33799 } }
I20260925 07:25:27.484710 9355 leader_election.cc:302] T 00000000000000000000000000000000 P 14ab1951295941e9b8f4c89200325711 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 14ab1951295941e9b8f4c89200325711; no voters:
I20260925 07:25:27.484820 9355 leader_election.cc:288] T 00000000000000000000000000000000 P 14ab1951295941e9b8f4c89200325711 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 07:25:27.484886 9358 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 14ab1951295941e9b8f4c89200325711 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:25:27.485026 9355 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14ab1951295941e9b8f4c89200325711 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:25:27.485049 9358 raft_consensus.cc:686] T 00000000000000000000000000000000 P 14ab1951295941e9b8f4c89200325711 [term 1 LEADER]: Becoming Leader. State: Replica: 14ab1951295941e9b8f4c89200325711, State: Running, Role: LEADER
I20260925 07:25:27.485129 9358 consensus_queue.cc:237] T 00000000000000000000000000000000 P 14ab1951295941e9b8f4c89200325711 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14ab1951295941e9b8f4c89200325711" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 33799 } }
I20260925 07:25:27.485498 9360 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14ab1951295941e9b8f4c89200325711 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 14ab1951295941e9b8f4c89200325711. Latest consensus state: current_term: 1 leader_uuid: "14ab1951295941e9b8f4c89200325711" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14ab1951295941e9b8f4c89200325711" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 33799 } } }
I20260925 07:25:27.485517 9359 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14ab1951295941e9b8f4c89200325711 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "14ab1951295941e9b8f4c89200325711" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14ab1951295941e9b8f4c89200325711" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 33799 } } }
I20260925 07:25:27.485674 9360 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14ab1951295941e9b8f4c89200325711 [sys.catalog]: This master's current role is: LEADER
I20260925 07:25:27.485687 9359 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14ab1951295941e9b8f4c89200325711 [sys.catalog]: This master's current role is: LEADER
I20260925 07:25:27.485991 9367 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 07:25:27.486374 9367 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 07:25:27.487712 9367 catalog_manager.cc:1347] Generated new cluster ID: 6ea5649da4c04e929df674f6971e904c
I20260925 07:25:27.487769 9367 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 07:25:27.487903 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 9287
I20260925 07:25:27.487994 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/master-0/wal/instance
I20260925 07:25:27.488147 5753 external_mini_cluster.cc:1428] Setting key 5ef0a50f994b142644ce3b7f634ce684
Sep 25 07:25:27 dist-test-slave-1drq krb5kdc[9274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:25:27.496107 9367 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 07:25:27.496546 9367 catalog_manager.cc:1504] Loading token signing keys...
I20260925 07:25:27.498432 9378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:27.488707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56220 (local address 127.5.158.126:33799)
0925 07:25:27.488875 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:27.488883 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:27.488910 (+ 27us) server_negotiation.cc:407] Connection header received
0925 07:25:27.489012 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:27.489018 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:27.489079 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:27.489209 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:27.489952 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:27.491192 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:27.492123 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:27.492651 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:27.492731 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:27.495054 (+ 2323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:27.495058 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:27.495058 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:27.495067 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:27.496921 (+ 1854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:27.497280 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:27.497282 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:27.497283 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:27.497376 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:27.497584 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:27.497585 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:27.497585 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:27.497852 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:27.498011 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:27.498081 (+ 70us) server_negotiation.cc:299] Negotiation successful
0925 07:25:27.498216 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":96,"threads_started":1}
I20260925 07:25:27.500789 9367 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 14ab1951295941e9b8f4c89200325711: Generated new TSK 0
I20260925 07:25:27.500946 9367 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.158.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab.
Entry for principal kudu/127.5.158.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab.
I20260925 07:25:27.525275 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.65:0
--local_ip_for_outbound_sockets=127.5.158.65
--webserver_interface=127.5.158.65
--webserver_port=0
--tserver_master_addrs=127.5.158.126:33799
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab
--principal=kudu/127.5.158.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:45383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:25:27.600788 9381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:25:27.600992 9381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:25:27.601024 9381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:25:27.602087 9381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:25:27.602145 9381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:25:27.602183 9381 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.65
I20260925 07:25:27.603485 9381 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:45383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab
--principal=kudu/127.5.158.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.158.65
--webserver_port=0
--tserver_master_addrs=127.5.158.126:33799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9381
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.65
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:25:27.603704 9381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:25:27.603890 9381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:25:27.608315 9387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:27.608319 9386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:27.608319 9389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:27.608556 9381 server_base.cc:1034] running on GCE node
I20260925 07:25:27.608784 9381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:25:27.609041 9381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:25:27.610213 9381 hybrid_clock.cc:648] HybridClock initialized: now 1790321127610194 us; error 33 us; skew 500 ppm
Sep 25 07:25:27 dist-test-slave-1drq krb5kdc[9274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321127, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:25:27.612501 9381 init.cc:377] Logged in from keytab as kudu/127.5.158.65@KRBTEST.COM (short username kudu)
I20260925 07:25:27.612907 9381 webserver.cc:460] Webserver started at http://127.5.158.65:36023/ using document root <none> and password file <none>
I20260925 07:25:27.613154 9381 fs_manager.cc:359] Metadata directory not provided
I20260925 07:25:27.613215 9381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:25:27.613338 9381 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:25:27.614238 9381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/ts-0/data/instance:
uuid: "f3510209f0f34ad4b58ed14b9c243d86"
format_stamp: "Formatted at 2026-09-25 07:25:27 on dist-test-slave-1drq"
server_key: "c9f43b61b7ec95b8cdb7ab867e4f781f"
server_key_iv: "e90838aedf5c1185b52091f6c7131421"
server_key_version: "encryptionkey@0"
I20260925 07:25:27.614567 9381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/ts-0/wal/instance:
uuid: "f3510209f0f34ad4b58ed14b9c243d86"
format_stamp: "Formatted at 2026-09-25 07:25:27 on dist-test-slave-1drq"
server_key: "c9f43b61b7ec95b8cdb7ab867e4f781f"
server_key_iv: "e90838aedf5c1185b52091f6c7131421"
server_key_version: "encryptionkey@0"
I20260925 07:25:27.615743 9381 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 07:25:27.616397 9396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:27.616551 9381 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 07:25:27.616636 9381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/ts-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/ts-0/wal
uuid: "f3510209f0f34ad4b58ed14b9c243d86"
format_stamp: "Formatted at 2026-09-25 07:25:27 on dist-test-slave-1drq"
server_key: "c9f43b61b7ec95b8cdb7ab867e4f781f"
server_key_iv: "e90838aedf5c1185b52091f6c7131421"
server_key_version: "encryptionkey@0"
I20260925 07:25:27.616710 9381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:25:27.637295 9381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:25:27.638933 9381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:25:27.639062 9381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:25:27.639254 9381 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:25:27.639523 9381 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:25:27.639556 9381 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:27.639575 9381 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:25:27.639585 9381 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:27.644377 9381 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.65:43407
I20260925 07:25:27.644435 9509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.65:43407 every 8 connection(s)
I20260925 07:25:27.644678 9381 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
I20260925 07:25:27.649396 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 9381
I20260925 07:25:27.649506 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/ts-0/wal/instance
I20260925 07:25:27.649688 5753 external_mini_cluster.cc:1428] Setting key e3de114b9dc6bf92e79d81ac54655235
Sep 25 07:25:27 dist-test-slave-1drq krb5kdc[9274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321127, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.65@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:25:27.653388 9378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:27.645302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:50087 (local address 127.5.158.126:33799)
0925 07:25:27.645341 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:27.645342 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:27.645521 (+ 179us) server_negotiation.cc:407] Connection header received
0925 07:25:27.646327 (+ 806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:27.646329 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:27.646339 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:27.646393 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:27.646913 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:27.647422 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:27.648402 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:27.648691 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:27.648756 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:27.650787 (+ 2031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:27.650788 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:27.650789 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:27.650795 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:27.652076 (+ 1281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:27.652490 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:27.652491 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:27.652491 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:27.652543 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:27.652776 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:27.652777 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:27.652777 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:27.652877 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:27.652934 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:27.653208 (+ 274us) server_negotiation.cc:299] Negotiation successful
0925 07:25:27.653306 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 07:25:27.653502 9512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:27.645281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:33799 (local address 127.5.158.65:50087)
0925 07:25:27.645422 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:27.645450 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:27.646231 (+ 781us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:27.646451 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:27.646455 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:27.646673 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:27.646824 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:27.646828 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:27.647524 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:27.647525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:27.648317 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:27.648319 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:27.648763 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:27.648764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:27.648875 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:27.649298 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:27.649307 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:27.650694 (+ 1387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:27.652166 (+ 1472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:27.652190 (+ 24us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:27.652191 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:27.652428 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:27.652605 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:27.652606 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:27.652606 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:27.652704 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:27.652937 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:27.652943 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:27.653063 (+ 120us) client_negotiation.cc:769] Sending connection context
0925 07:25:27.653178 (+ 115us) client_negotiation.cc:240] Negotiation successful
0925 07:25:27.653313 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":68,"threads_started":1}
I20260925 07:25:27.653915 9510 heartbeater.cc:347] Connected to a master server at 127.5.158.126:33799
I20260925 07:25:27.653990 9510 heartbeater.cc:464] Registering TS with master...
I20260925 07:25:27.654145 9510 heartbeater.cc:511] Master 127.5.158.126:33799 requested a full tablet report, sending...
I20260925 07:25:27.654563 9319 ts_manager.cc:194] Registered new tserver with Master: f3510209f0f34ad4b58ed14b9c243d86 (127.5.158.65:43407)
I20260925 07:25:27.655064 9319 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.65@KRBTEST.COM'} at 127.5.158.65:50087
WARNING: no policy specified for kudu/127.5.158.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab.
Entry for principal kudu/127.5.158.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab.
I20260925 07:25:27.672940 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.66:0
--local_ip_for_outbound_sockets=127.5.158.66
--webserver_interface=127.5.158.66
--webserver_port=0
--tserver_master_addrs=127.5.158.126:33799
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab
--principal=kudu/127.5.158.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:45383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:25:27.741974 9515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:25:27.742123 9515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:25:27.742142 9515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:25:27.743036 9515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:25:27.743062 9515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:25:27.743084 9515 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.66
I20260925 07:25:27.744256 9515 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:45383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab
--principal=kudu/127.5.158.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.158.66
--webserver_port=0
--tserver_master_addrs=127.5.158.126:33799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9515
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.66
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:25:27.744393 9515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:25:27.744524 9515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:25:27.748963 9521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:27.748943 9520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:27.748999 9515 server_base.cc:1034] running on GCE node
W20260925 07:25:27.748956 9523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:27.749351 9515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:25:27.749516 9515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:25:27.750641 9515 hybrid_clock.cc:648] HybridClock initialized: now 1790321127750625 us; error 33 us; skew 500 ppm
Sep 25 07:25:27 dist-test-slave-1drq krb5kdc[9274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321127, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:25:27.752645 9515 init.cc:377] Logged in from keytab as kudu/127.5.158.66@KRBTEST.COM (short username kudu)
I20260925 07:25:27.753015 9515 webserver.cc:460] Webserver started at http://127.5.158.66:42733/ using document root <none> and password file <none>
I20260925 07:25:27.753234 9515 fs_manager.cc:359] Metadata directory not provided
I20260925 07:25:27.753274 9515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:25:27.753350 9515 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:25:27.754200 9515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/ts-1/data/instance:
uuid: "acf2e637fa414292832784b6bd9eabdc"
format_stamp: "Formatted at 2026-09-25 07:25:27 on dist-test-slave-1drq"
server_key: "2bb81174635e6b708fbb18c9853b5f54"
server_key_iv: "de1a6a467fc2e1f5ec3c393446a9254f"
server_key_version: "encryptionkey@0"
I20260925 07:25:27.754452 9515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/ts-1/wal/instance:
uuid: "acf2e637fa414292832784b6bd9eabdc"
format_stamp: "Formatted at 2026-09-25 07:25:27 on dist-test-slave-1drq"
server_key: "2bb81174635e6b708fbb18c9853b5f54"
server_key_iv: "de1a6a467fc2e1f5ec3c393446a9254f"
server_key_version: "encryptionkey@0"
I20260925 07:25:27.755494 9515 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 07:25:27.756021 9530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:27.756145 9515 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:25:27.756210 9515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/ts-1/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/ts-1/wal
uuid: "acf2e637fa414292832784b6bd9eabdc"
format_stamp: "Formatted at 2026-09-25 07:25:27 on dist-test-slave-1drq"
server_key: "2bb81174635e6b708fbb18c9853b5f54"
server_key_iv: "de1a6a467fc2e1f5ec3c393446a9254f"
server_key_version: "encryptionkey@0"
I20260925 07:25:27.756261 9515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:25:27.767175 9515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:25:27.768800 9515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:25:27.768883 9515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:25:27.769008 9515 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:25:27.769266 9515 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:25:27.769294 9515 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:27.769311 9515 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:25:27.769348 9515 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:27.774684 9515 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.66:40035
I20260925 07:25:27.774843 9643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.66:40035 every 8 connection(s)
I20260925 07:25:27.775013 9515 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
I20260925 07:25:27.776698 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 9515
I20260925 07:25:27.776815 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/ts-1/wal/instance
I20260925 07:25:27.776991 5753 external_mini_cluster.cc:1428] Setting key 01923b5e4974415aa59132e3af11757e
Sep 25 07:25:27 dist-test-slave-1drq krb5kdc[9274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321127, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.66@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.158.67@KRBTEST.COM; defaulting to no policy
I20260925 07:25:27.786372 9378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:27.776055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:41079 (local address 127.5.158.126:33799)
0925 07:25:27.776103 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:27.776104 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:27.776938 (+ 834us) server_negotiation.cc:407] Connection header received
0925 07:25:27.778524 (+ 1586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:27.778525 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:27.778538 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:27.778596 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:27.779431 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:27.780146 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:27.781065 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:27.781397 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:27.781482 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:27.783728 (+ 2246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:27.783729 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:27.783729 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:27.783736 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:27.785014 (+ 1278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:27.785490 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:27.785491 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:27.785491 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:27.785539 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:27.785809 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:27.785810 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:27.785810 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:27.785904 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:27.785952 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:27.786256 (+ 304us) server_negotiation.cc:299] Negotiation successful
0925 07:25:27.786314 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260925 07:25:27.786597 9646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:27.775764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:33799 (local address 127.5.158.66:41079)
0925 07:25:27.776814 (+ 1050us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:27.776856 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:27.778345 (+ 1489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:27.778746 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:27.778750 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:27.779092 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:27.779344 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:27.779347 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:27.780246 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:27.780247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:27.780980 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:27.780981 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:27.781494 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:27.781495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:27.781585 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:27.782051 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:27.782058 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:27.783627 (+ 1569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:27.785112 (+ 1485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:27.785118 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:27.785120 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:27.785423 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:27.785610 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:27.785611 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:27.785612 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:27.785754 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:27.785962 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:27.785967 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:27.786121 (+ 154us) client_negotiation.cc:769] Sending connection context
0925 07:25:27.786236 (+ 115us) client_negotiation.cc:240] Negotiation successful
0925 07:25:27.786386 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1008,"thread_start_us":84,"threads_started":1}
I20260925 07:25:27.787022 9644 heartbeater.cc:347] Connected to a master server at 127.5.158.126:33799
I20260925 07:25:27.787114 9644 heartbeater.cc:464] Registering TS with master...
I20260925 07:25:27.787258 9644 heartbeater.cc:511] Master 127.5.158.126:33799 requested a full tablet report, sending...
I20260925 07:25:27.787616 9319 ts_manager.cc:194] Registered new tserver with Master: acf2e637fa414292832784b6bd9eabdc (127.5.158.66:40035)
I20260925 07:25:27.788007 9319 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.66@KRBTEST.COM'} at 127.5.158.66:41079
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab.
Entry for principal kudu/127.5.158.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab.
I20260925 07:25:27.801662 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.67:0
--local_ip_for_outbound_sockets=127.5.158.67
--webserver_interface=127.5.158.67
--webserver_port=0
--tserver_master_addrs=127.5.158.126:33799
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab
--principal=kudu/127.5.158.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:45383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:25:27.874636 9649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:25:27.874791 9649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:25:27.874840 9649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:25:27.875836 9649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:25:27.875872 9649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:25:27.875893 9649 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.67
I20260925 07:25:27.877331 9649 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:45383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab
--principal=kudu/127.5.158.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.158.67
--webserver_port=0
--tserver_master_addrs=127.5.158.126:33799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9649
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.67
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:25:27.877565 9649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:25:27.877763 9649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:25:27.881924 9654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:27.881920 9655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:27.881919 9657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:27.882104 9649 server_base.cc:1034] running on GCE node
I20260925 07:25:27.882304 9649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:25:27.882515 9649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:25:27.883657 9649 hybrid_clock.cc:648] HybridClock initialized: now 1790321127883637 us; error 37 us; skew 500 ppm
Sep 25 07:25:27 dist-test-slave-1drq krb5kdc[9274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321127, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:25:27.886005 9649 init.cc:377] Logged in from keytab as kudu/127.5.158.67@KRBTEST.COM (short username kudu)
I20260925 07:25:27.886394 9649 webserver.cc:460] Webserver started at http://127.5.158.67:41209/ using document root <none> and password file <none>
I20260925 07:25:27.886611 9649 fs_manager.cc:359] Metadata directory not provided
I20260925 07:25:27.886652 9649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:25:27.886780 9649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:25:27.887699 9649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/ts-2/data/instance:
uuid: "894bed0ea06143c49ca4e13754667014"
format_stamp: "Formatted at 2026-09-25 07:25:27 on dist-test-slave-1drq"
server_key: "7d8867664cbb3e82848bd33c5adb31f0"
server_key_iv: "d95ab697e39aaed42940d55da896dad9"
server_key_version: "encryptionkey@0"
I20260925 07:25:27.888024 9649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/ts-2/wal/instance:
uuid: "894bed0ea06143c49ca4e13754667014"
format_stamp: "Formatted at 2026-09-25 07:25:27 on dist-test-slave-1drq"
server_key: "7d8867664cbb3e82848bd33c5adb31f0"
server_key_iv: "d95ab697e39aaed42940d55da896dad9"
server_key_version: "encryptionkey@0"
I20260925 07:25:27.889312 9649 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 07:25:27.889962 9664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:27.890116 9649 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:25:27.890199 9649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/ts-2/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/ts-2/wal
uuid: "894bed0ea06143c49ca4e13754667014"
format_stamp: "Formatted at 2026-09-25 07:25:27 on dist-test-slave-1drq"
server_key: "7d8867664cbb3e82848bd33c5adb31f0"
server_key_iv: "d95ab697e39aaed42940d55da896dad9"
server_key_version: "encryptionkey@0"
I20260925 07:25:27.890250 9649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:25:27.905035 9649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:25:27.907114 9649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:25:27.907214 9649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:25:27.907366 9649 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:25:27.907593 9649 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:25:27.907622 9649 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:27.907641 9649 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:25:27.907655 9649 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:27.913674 9649 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.67:33127
I20260925 07:25:27.913726 9777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.67:33127 every 8 connection(s)
I20260925 07:25:27.913992 9649 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
I20260925 07:25:27.915369 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 9649
I20260925 07:25:27.915489 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790321110323796-5753-0/minicluster-data/ts-2/wal/instance
I20260925 07:25:27.915649 5753 external_mini_cluster.cc:1428] Setting key 57a24d4c669114a8aea1f91670f11bda
Sep 25 07:25:27 dist-test-slave-1drq krb5kdc[9274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321127, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.67@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:25:27.922464 9378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:27.914618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:54765 (local address 127.5.158.126:33799)
0925 07:25:27.914670 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:27.914673 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:27.914870 (+ 197us) server_negotiation.cc:407] Connection header received
0925 07:25:27.915820 (+ 950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:27.915822 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:27.915836 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:27.915895 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:27.916639 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:27.917201 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:27.918159 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:27.918347 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:27.918396 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:27.920355 (+ 1959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:27.920356 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:27.920356 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:27.920362 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:27.921447 (+ 1085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:27.921798 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:27.921798 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:27.921799 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:27.921844 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:27.922028 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:27.922028 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:27.922028 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:27.922114 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:27.922159 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:27.922352 (+ 193us) server_negotiation.cc:299] Negotiation successful
0925 07:25:27.922407 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 07:25:27.922667 9780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:27.914603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:33799 (local address 127.5.158.67:54765)
0925 07:25:27.914759 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:27.914791 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:27.915703 (+ 912us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:27.915972 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:27.915975 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:27.916318 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:27.916551 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:27.916557 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:27.917281 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:27.917281 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:27.918085 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:27.918087 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:27.918435 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:27.918436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:27.918538 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:27.919006 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:27.919013 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:27.920285 (+ 1272us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:27.921522 (+ 1237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:27.921528 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:27.921529 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:27.921733 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:27.921896 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:27.921896 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:27.921896 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:27.921981 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:27.922161 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:27.922164 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:27.922258 (+ 94us) client_negotiation.cc:769] Sending connection context
0925 07:25:27.922331 (+ 73us) client_negotiation.cc:240] Negotiation successful
0925 07:25:27.922440 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":82,"threads_started":1}
I20260925 07:25:27.923105 9778 heartbeater.cc:347] Connected to a master server at 127.5.158.126:33799
I20260925 07:25:27.923192 9778 heartbeater.cc:464] Registering TS with master...
I20260925 07:25:27.923348 9778 heartbeater.cc:511] Master 127.5.158.126:33799 requested a full tablet report, sending...
I20260925 07:25:27.923669 9319 ts_manager.cc:194] Registered new tserver with Master: 894bed0ea06143c49ca4e13754667014 (127.5.158.67:33127)
I20260925 07:25:27.924033 9319 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.67@KRBTEST.COM'} at 127.5.158.67:54765
I20260925 07:25:27.927232 5753 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 25 07:25:27 dist-test-slave-1drq krb5kdc[9274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 07:25:27 dist-test-slave-1drq krb5kdc[9274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.158.65@KRBTEST.COM
I20260925 07:25:27.947649 9785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:27.940948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34198 (local address 127.5.158.65:43407)
0925 07:25:27.941172 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:27.941176 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:27.941186 (+ 10us) server_negotiation.cc:407] Connection header received
0925 07:25:27.941207 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:27.941209 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:27.941221 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:27.941292 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:27.941772 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:27.942435 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:27.943036 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:27.943306 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:27.943352 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:27.945263 (+ 1911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:27.945269 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:27.945270 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:27.945276 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:27.946716 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:27.946989 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:27.946991 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:27.946992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:27.947040 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:27.947169 (+ 129us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:27.947170 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:27.947170 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:27.947288 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:27.947400 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:27.947455 (+ 55us) server_negotiation.cc:299] Negotiation successful
0925 07:25:27.947530 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":86,"threads_started":1}
Sep 25 07:25:27 dist-test-slave-1drq krb5kdc[9274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:25:27.954272 9378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:27.948715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56228 (local address 127.5.158.126:33799)
0925 07:25:27.948745 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:27.948747 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:27.948778 (+ 31us) server_negotiation.cc:407] Connection header received
0925 07:25:27.948826 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:27.948827 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:27.948833 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:27.948875 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:27.949364 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:27.949747 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:27.950292 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:27.950447 (+ 155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:27.950487 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:27.952383 (+ 1896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:27.952384 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:27.952384 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:27.952389 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:27.953473 (+ 1084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:27.953747 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:27.953748 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:27.953748 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:27.953791 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:27.953911 (+ 120us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:27.953912 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:27.953912 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:27.954024 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:27.954095 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:27.954168 (+ 73us) server_negotiation.cc:299] Negotiation successful
0925 07:25:27.954215 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
W20260925 07:25:27.954435 9319 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56228
I20260925 07:25:27.954686 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 9381
I20260925 07:25:27.959349 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 9515
I20260925 07:25:27.963752 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 9649
I20260925 07:25:27.967549 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 9287
2026-09-25T07:25:27Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1638 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 07:25:28 dist-test-slave-1drq krb5kdc[9790](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 07:25:28 dist-test-slave-1drq krb5kdc[9790](info): set up 2 sockets
Sep 25 07:25:28 dist-test-slave-1drq krb5kdc[9790](info): commencing operation
krb5kdc: starting...
W20260925 07:25:28.900734 5753 mini_kdc.cc:121] Time spent starting KDC: real 0.914s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Sep 25 07:25:28 dist-test-slave-1drq krb5kdc[9790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321128, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T07:25:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T07:25:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
I20260925 07:25:29.015892 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:42707
--webserver_interface=127.5.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:33743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.158.126:42707
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:25:29.089308 9803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:25:29.089521 9803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:25:29.089553 9803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:25:29.090557 9803 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:25:29.090615 9803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:25:29.090636 9803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:25:29.090656 9803 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:25:29.090680 9803 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:25:29.091969 9803 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:33743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.158.126:42707
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:42707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9803
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:25:29.092219 9803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:25:29.092394 9803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:25:29.096427 9808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:29.096423 9809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:29.096485 9803 server_base.cc:1034] running on GCE node
W20260925 07:25:29.096427 9811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:29.096858 9803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:25:29.097136 9803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:25:29.098280 9803 hybrid_clock.cc:648] HybridClock initialized: now 1790321129098268 us; error 25 us; skew 500 ppm
Sep 25 07:25:29 dist-test-slave-1drq krb5kdc[9790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321129, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:25:29.100468 9803 init.cc:377] Logged in from keytab as kudu/127.5.158.126@KRBTEST.COM (short username kudu)
I20260925 07:25:29.100750 9803 fs_manager.cc:359] Metadata directory not provided
I20260925 07:25:29.100790 9803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:25:29.100929 9803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:25:29.101842 9803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/master-0/data/instance:
uuid: "6a2a302f58dc4b059e7dd6c2dfb81057"
format_stamp: "Formatted at 2026-09-25 07:25:29 on dist-test-slave-1drq"
server_key: "90734358fef6a8d35444e2c24a28a379"
server_key_iv: "b4210d70fa203144791caf6d569bad55"
server_key_version: "encryptionkey@0"
I20260925 07:25:29.102169 9803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/master-0/wal/instance:
uuid: "6a2a302f58dc4b059e7dd6c2dfb81057"
format_stamp: "Formatted at 2026-09-25 07:25:29 on dist-test-slave-1drq"
server_key: "90734358fef6a8d35444e2c24a28a379"
server_key_iv: "b4210d70fa203144791caf6d569bad55"
server_key_version: "encryptionkey@0"
I20260925 07:25:29.103268 9803 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260925 07:25:29.103832 9817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:29.103960 9803 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:29.104035 9803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "6a2a302f58dc4b059e7dd6c2dfb81057"
format_stamp: "Formatted at 2026-09-25 07:25:29 on dist-test-slave-1drq"
server_key: "90734358fef6a8d35444e2c24a28a379"
server_key_iv: "b4210d70fa203144791caf6d569bad55"
server_key_version: "encryptionkey@0"
I20260925 07:25:29.104081 9803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:25:29.114279 9803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:25:29.116178 9803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:25:29.116290 9803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:25:29.119776 9803 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.126:42707
I20260925 07:25:29.119799 9869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.126:42707 every 8 connection(s)
I20260925 07:25:29.120040 9803 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:25:29.120801 9870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:25:29.122714 9870 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a2a302f58dc4b059e7dd6c2dfb81057: Bootstrap starting.
I20260925 07:25:29.123232 9870 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6a2a302f58dc4b059e7dd6c2dfb81057: Neither blocks nor log segments found. Creating new log.
I20260925 07:25:29.123502 9870 log.cc:824] T 00000000000000000000000000000000 P 6a2a302f58dc4b059e7dd6c2dfb81057: Log is configured to *not* fsync() on all Append() calls
I20260925 07:25:29.124095 9870 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a2a302f58dc4b059e7dd6c2dfb81057: No bootstrap required, opened a new log
I20260925 07:25:29.125649 9870 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a2a302f58dc4b059e7dd6c2dfb81057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a2a302f58dc4b059e7dd6c2dfb81057" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 42707 } }
I20260925 07:25:29.125759 9870 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6a2a302f58dc4b059e7dd6c2dfb81057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:25:29.125797 9870 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a2a302f58dc4b059e7dd6c2dfb81057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a2a302f58dc4b059e7dd6c2dfb81057, State: Initialized, Role: FOLLOWER
I20260925 07:25:29.125922 9870 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a2a302f58dc4b059e7dd6c2dfb81057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a2a302f58dc4b059e7dd6c2dfb81057" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 42707 } }
I20260925 07:25:29.125973 9870 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6a2a302f58dc4b059e7dd6c2dfb81057 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 07:25:29.125991 9870 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6a2a302f58dc4b059e7dd6c2dfb81057 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 07:25:29.126050 9870 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a2a302f58dc4b059e7dd6c2dfb81057 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:25:29.126734 9870 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6a2a302f58dc4b059e7dd6c2dfb81057 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a2a302f58dc4b059e7dd6c2dfb81057" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 42707 } }
I20260925 07:25:29.126833 9870 leader_election.cc:302] T 00000000000000000000000000000000 P 6a2a302f58dc4b059e7dd6c2dfb81057 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6a2a302f58dc4b059e7dd6c2dfb81057; no voters:
I20260925 07:25:29.126995 9870 leader_election.cc:288] T 00000000000000000000000000000000 P 6a2a302f58dc4b059e7dd6c2dfb81057 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 07:25:29.127033 9873 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6a2a302f58dc4b059e7dd6c2dfb81057 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:25:29.127192 9873 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6a2a302f58dc4b059e7dd6c2dfb81057 [term 1 LEADER]: Becoming Leader. State: Replica: 6a2a302f58dc4b059e7dd6c2dfb81057, State: Running, Role: LEADER
I20260925 07:25:29.127219 9870 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a2a302f58dc4b059e7dd6c2dfb81057 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:25:29.127271 9873 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a2a302f58dc4b059e7dd6c2dfb81057 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a2a302f58dc4b059e7dd6c2dfb81057" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 42707 } }
I20260925 07:25:29.127652 9874 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a2a302f58dc4b059e7dd6c2dfb81057 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6a2a302f58dc4b059e7dd6c2dfb81057" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a2a302f58dc4b059e7dd6c2dfb81057" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 42707 } } }
I20260925 07:25:29.127653 9875 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a2a302f58dc4b059e7dd6c2dfb81057 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a2a302f58dc4b059e7dd6c2dfb81057. Latest consensus state: current_term: 1 leader_uuid: "6a2a302f58dc4b059e7dd6c2dfb81057" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a2a302f58dc4b059e7dd6c2dfb81057" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 42707 } } }
I20260925 07:25:29.127784 9874 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a2a302f58dc4b059e7dd6c2dfb81057 [sys.catalog]: This master's current role is: LEADER
I20260925 07:25:29.127823 9875 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a2a302f58dc4b059e7dd6c2dfb81057 [sys.catalog]: This master's current role is: LEADER
I20260925 07:25:29.128060 9882 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 07:25:29.128530 9882 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 07:25:29.129819 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 9803
I20260925 07:25:29.129935 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/master-0/wal/instance
I20260925 07:25:29.130103 5753 external_mini_cluster.cc:1428] Setting key ba596972d4dc82f97e6ec8e860028953
I20260925 07:25:29.130328 9882 catalog_manager.cc:1347] Generated new cluster ID: 7e084ec6dd8b4427bf3c7b255addb369
I20260925 07:25:29.130376 9882 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 07:25:29 dist-test-slave-1drq krb5kdc[9790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321128, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:25:29.140540 9893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:29.130746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35576 (local address 127.5.158.126:42707)
0925 07:25:29.130884 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:29.130892 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:29.130920 (+ 28us) server_negotiation.cc:407] Connection header received
0925 07:25:29.131026 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:29.131031 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:29.131077 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:29.131200 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:29.131804 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:29.133076 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:29.133988 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:29.134643 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:29.134735 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:29.137227 (+ 2492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:29.137231 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:29.137232 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:29.137242 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:29.138991 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:29.139339 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:29.139341 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:29.139342 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:29.139426 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:29.139603 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:29.139604 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:29.139604 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:29.139870 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:29.139993 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:29.140123 (+ 130us) server_negotiation.cc:299] Negotiation successful
0925 07:25:29.140323 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":63,"threads_started":1}
I20260925 07:25:29.151644 9882 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 07:25:29.152094 9882 catalog_manager.cc:1504] Loading token signing keys...
I20260925 07:25:29.155411 9882 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6a2a302f58dc4b059e7dd6c2dfb81057: Generated new TSK 0
I20260925 07:25:29.155575 9882 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.158.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab.
Entry for principal kudu/127.5.158.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab.
I20260925 07:25:29.183142 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.65:0
--local_ip_for_outbound_sockets=127.5.158.65
--webserver_interface=127.5.158.65
--webserver_port=0
--tserver_master_addrs=127.5.158.126:42707
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab
--principal=kudu/127.5.158.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:33743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:25:29.253731 9896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:25:29.253896 9896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:25:29.253916 9896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:25:29.254930 9896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:25:29.254979 9896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:25:29.255002 9896 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.65
I20260925 07:25:29.256311 9896 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:33743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab
--principal=kudu/127.5.158.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.158.65
--webserver_port=0
--tserver_master_addrs=127.5.158.126:42707
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9896
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.65
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:25:29.256471 9896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:25:29.256609 9896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:25:29.260927 9904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:29.260973 9902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:29.261030 9896 server_base.cc:1034] running on GCE node
W20260925 07:25:29.260927 9901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:29.261305 9896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:25:29.261520 9896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:25:29.262676 9896 hybrid_clock.cc:648] HybridClock initialized: now 1790321129262672 us; error 31 us; skew 500 ppm
Sep 25 07:25:29 dist-test-slave-1drq krb5kdc[9790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321129, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:25:29.264803 9896 init.cc:377] Logged in from keytab as kudu/127.5.158.65@KRBTEST.COM (short username kudu)
I20260925 07:25:29.265076 9896 fs_manager.cc:359] Metadata directory not provided
I20260925 07:25:29.265185 9896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:25:29.265303 9896 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:25:29.266149 9896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/ts-0/data/instance:
uuid: "3cd60fa02f094b3a94ebc217a1ec6b3b"
format_stamp: "Formatted at 2026-09-25 07:25:29 on dist-test-slave-1drq"
server_key: "a3e1ee6c746db9317043442c9458bf5d"
server_key_iv: "dfa3e4535a424b0bc0175563bbaf25b7"
server_key_version: "encryptionkey@0"
I20260925 07:25:29.266443 9896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/ts-0/wal/instance:
uuid: "3cd60fa02f094b3a94ebc217a1ec6b3b"
format_stamp: "Formatted at 2026-09-25 07:25:29 on dist-test-slave-1drq"
server_key: "a3e1ee6c746db9317043442c9458bf5d"
server_key_iv: "dfa3e4535a424b0bc0175563bbaf25b7"
server_key_version: "encryptionkey@0"
I20260925 07:25:29.267599 9896 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 07:25:29.268200 9910 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:29.268350 9896 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:25:29.268437 9896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/ts-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/ts-0/wal
uuid: "3cd60fa02f094b3a94ebc217a1ec6b3b"
format_stamp: "Formatted at 2026-09-25 07:25:29 on dist-test-slave-1drq"
server_key: "a3e1ee6c746db9317043442c9458bf5d"
server_key_iv: "dfa3e4535a424b0bc0175563bbaf25b7"
server_key_version: "encryptionkey@0"
I20260925 07:25:29.268512 9896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:25:29.282018 9896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:25:29.283443 9896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:25:29.283617 9896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:25:29.283790 9896 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:25:29.284031 9896 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:25:29.284075 9896 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:29.284101 9896 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:25:29.284126 9896 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:29.289328 9896 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.65:44525
I20260925 07:25:29.289371 10023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.65:44525 every 8 connection(s)
I20260925 07:25:29.289564 9896 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
Sep 25 07:25:29 dist-test-slave-1drq krb5kdc[9790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321129, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.65@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:25:29.296832 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 9896
I20260925 07:25:29.296881 9893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:29.290109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:57869 (local address 127.5.158.126:42707)
0925 07:25:29.290147 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:29.290149 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:29.290322 (+ 173us) server_negotiation.cc:407] Connection header received
0925 07:25:29.290681 (+ 359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:29.290682 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:29.290693 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:29.290742 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:29.291481 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:29.291920 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:29.292743 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:29.292920 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:29.292962 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:29.294672 (+ 1710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:29.294673 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:29.294674 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:29.294682 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:29.295764 (+ 1082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:29.296090 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:29.296091 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:29.296091 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:29.296143 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:29.296322 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:29.296323 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:29.296323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:29.296425 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:29.296487 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:29.296703 (+ 216us) server_negotiation.cc:299] Negotiation successful
0925 07:25:29.296785 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 07:25:29.296881 10026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:29.290072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42707 (local address 127.5.158.65:57869)
0925 07:25:29.290216 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:29.290250 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:29.290612 (+ 362us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:29.290803 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:29.290806 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:29.291010 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:29.291412 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:29.291416 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:29.291985 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:29.291986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:29.292681 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:29.292682 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:29.292978 (+ 296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:29.292978 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:29.293057 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:29.293463 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:29.293471 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:29.294593 (+ 1122us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:29.295825 (+ 1232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:29.295831 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:29.295832 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:29.296032 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:29.296193 (+ 161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:29.296193 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:29.296194 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:29.296276 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:29.296488 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:29.296492 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:29.296583 (+ 91us) client_negotiation.cc:769] Sending connection context
0925 07:25:29.296656 (+ 73us) client_negotiation.cc:240] Negotiation successful
0925 07:25:29.296753 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":83,"threads_started":1}
I20260925 07:25:29.296947 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/ts-0/wal/instance
I20260925 07:25:29.297159 5753 external_mini_cluster.cc:1428] Setting key 89cbc4465e47931b5a696e06be729577
I20260925 07:25:29.297379 10024 heartbeater.cc:347] Connected to a master server at 127.5.158.126:42707
I20260925 07:25:29.297482 10024 heartbeater.cc:464] Registering TS with master...
I20260925 07:25:29.297631 10024 heartbeater.cc:511] Master 127.5.158.126:42707 requested a full tablet report, sending...
I20260925 07:25:29.298136 9834 ts_manager.cc:194] Registered new tserver with Master: 3cd60fa02f094b3a94ebc217a1ec6b3b (127.5.158.65:44525)
I20260925 07:25:29.299002 9834 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.65@KRBTEST.COM'} at 127.5.158.65:57869
WARNING: no policy specified for kudu/127.5.158.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab.
Entry for principal kudu/127.5.158.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab.
I20260925 07:25:29.319839 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.66:0
--local_ip_for_outbound_sockets=127.5.158.66
--webserver_interface=127.5.158.66
--webserver_port=0
--tserver_master_addrs=127.5.158.126:42707
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab
--principal=kudu/127.5.158.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:33743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:25:29.393950 10029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:25:29.394109 10029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:25:29.394127 10029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:25:29.395126 10029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:25:29.395159 10029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:25:29.395185 10029 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.66
I20260925 07:25:29.396502 10029 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:33743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab
--principal=kudu/127.5.158.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.158.66
--webserver_port=0
--tserver_master_addrs=127.5.158.126:42707
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10029
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.66
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:25:29.396656 10029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:25:29.396801 10029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:25:29.401254 10037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:29.401250 10035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:29.401250 10034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:29.401384 10029 server_base.cc:1034] running on GCE node
I20260925 07:25:29.401650 10029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:25:29.401860 10029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:25:29.403028 10029 hybrid_clock.cc:648] HybridClock initialized: now 1790321129403005 us; error 32 us; skew 500 ppm
Sep 25 07:25:29 dist-test-slave-1drq krb5kdc[9790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321129, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:25:29.405300 10029 init.cc:377] Logged in from keytab as kudu/127.5.158.66@KRBTEST.COM (short username kudu)
I20260925 07:25:29.405640 10029 fs_manager.cc:359] Metadata directory not provided
I20260925 07:25:29.405699 10029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:25:29.405817 10029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:25:29.406785 10029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/ts-1/data/instance:
uuid: "894bfc7462de40199e1cfa8f202e3f05"
format_stamp: "Formatted at 2026-09-25 07:25:29 on dist-test-slave-1drq"
server_key: "9da3de96662dd844b5edc39619afb104"
server_key_iv: "8be40461744bdb4aabcb866f78f6c7fb"
server_key_version: "encryptionkey@0"
I20260925 07:25:29.407095 10029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/ts-1/wal/instance:
uuid: "894bfc7462de40199e1cfa8f202e3f05"
format_stamp: "Formatted at 2026-09-25 07:25:29 on dist-test-slave-1drq"
server_key: "9da3de96662dd844b5edc39619afb104"
server_key_iv: "8be40461744bdb4aabcb866f78f6c7fb"
server_key_version: "encryptionkey@0"
I20260925 07:25:29.408334 10029 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 07:25:29.408981 10043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:29.409266 10029 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260925 07:25:29.409353 10029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/ts-1/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/ts-1/wal
uuid: "894bfc7462de40199e1cfa8f202e3f05"
format_stamp: "Formatted at 2026-09-25 07:25:29 on dist-test-slave-1drq"
server_key: "9da3de96662dd844b5edc39619afb104"
server_key_iv: "8be40461744bdb4aabcb866f78f6c7fb"
server_key_version: "encryptionkey@0"
I20260925 07:25:29.409435 10029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:25:29.428247 10029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:25:29.429965 10029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:25:29.430109 10029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:25:29.430315 10029 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:25:29.430649 10029 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:25:29.430701 10029 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:29.430733 10029 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:25:29.430760 10029 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:29.436072 10029 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.66:34095
I20260925 07:25:29.436117 10156 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.66:34095 every 8 connection(s)
I20260925 07:25:29.436384 10029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
Sep 25 07:25:29 dist-test-slave-1drq krb5kdc[9790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321129, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.66@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:25:29.443531 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 10029
I20260925 07:25:29.443631 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/ts-1/wal/instance
I20260925 07:25:29.443729 9893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:29.436996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:59561 (local address 127.5.158.126:42707)
0925 07:25:29.437057 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:29.437059 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:29.437245 (+ 186us) server_negotiation.cc:407] Connection header received
0925 07:25:29.437670 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:29.437671 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:29.437681 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:29.437721 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:29.438496 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:29.438940 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:29.439722 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:29.439876 (+ 154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:29.439915 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:29.441583 (+ 1668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:29.441584 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:29.441584 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:29.441590 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:29.442797 (+ 1207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:29.443078 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:29.443079 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:29.443079 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:29.443122 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:29.443250 (+ 128us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:29.443251 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:29.443251 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:29.443327 (+ 76us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:29.443364 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:29.443569 (+ 205us) server_negotiation.cc:299] Negotiation successful
0925 07:25:29.443651 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 07:25:29.443819 5753 external_mini_cluster.cc:1428] Setting key b789f4bc4c07f26e9fc7e9bc33859b2e
I20260925 07:25:29.443833 10159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:29.436986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42707 (local address 127.5.158.66:59561)
0925 07:25:29.437130 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:29.437164 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:29.437584 (+ 420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:29.437767 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:29.437770 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:29.437991 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:29.438429 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:29.438432 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:29.439006 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:29.439007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:29.439662 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:29.439664 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:29.439931 (+ 267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:29.439932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:29.440010 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:29.440323 (+ 313us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:29.440330 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:29.441510 (+ 1180us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:29.442852 (+ 1342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:29.442857 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:29.442858 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:29.443018 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:29.443157 (+ 139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:29.443157 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:29.443158 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:29.443222 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:29.443364 (+ 142us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:29.443366 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:29.443450 (+ 84us) client_negotiation.cc:769] Sending connection context
0925 07:25:29.443537 (+ 87us) client_negotiation.cc:240] Negotiation successful
0925 07:25:29.443655 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":67,"threads_started":1}
I20260925 07:25:29.444257 10157 heartbeater.cc:347] Connected to a master server at 127.5.158.126:42707
I20260925 07:25:29.444334 10157 heartbeater.cc:464] Registering TS with master...
I20260925 07:25:29.444448 10157 heartbeater.cc:511] Master 127.5.158.126:42707 requested a full tablet report, sending...
I20260925 07:25:29.444784 9833 ts_manager.cc:194] Registered new tserver with Master: 894bfc7462de40199e1cfa8f202e3f05 (127.5.158.66:34095)
I20260925 07:25:29.445147 9833 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.66@KRBTEST.COM'} at 127.5.158.66:59561
WARNING: no policy specified for kudu/127.5.158.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab.
Entry for principal kudu/127.5.158.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab.
I20260925 07:25:29.466053 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.67:0
--local_ip_for_outbound_sockets=127.5.158.67
--webserver_interface=127.5.158.67
--webserver_port=0
--tserver_master_addrs=127.5.158.126:42707
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab
--principal=kudu/127.5.158.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:33743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:25:29.538318 10162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:25:29.538492 10162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:25:29.538514 10162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:25:29.539630 10162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:25:29.539669 10162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:25:29.539695 10162 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.67
I20260925 07:25:29.541059 10162 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:33743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab
--principal=kudu/127.5.158.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.158.67
--webserver_port=0
--tserver_master_addrs=127.5.158.126:42707
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10162
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.67
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:25:29.541273 10162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:25:29.541460 10162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:25:29.545740 10167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:29.545734 10170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:29.545734 10168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:29.546000 10162 server_base.cc:1034] running on GCE node
I20260925 07:25:29.546144 10162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:25:29.546329 10162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:25:29.547482 10162 hybrid_clock.cc:648] HybridClock initialized: now 1790321129547472 us; error 24 us; skew 500 ppm
Sep 25 07:25:29 dist-test-slave-1drq krb5kdc[9790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321129, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:25:29.549719 10162 init.cc:377] Logged in from keytab as kudu/127.5.158.67@KRBTEST.COM (short username kudu)
I20260925 07:25:29.549993 10162 fs_manager.cc:359] Metadata directory not provided
I20260925 07:25:29.550033 10162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:25:29.550140 10162 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:25:29.551102 10162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/ts-2/data/instance:
uuid: "0d316d7f90fd42c887a931fae5fd2f42"
format_stamp: "Formatted at 2026-09-25 07:25:29 on dist-test-slave-1drq"
server_key: "234fb9f608e4defbd803e549df009069"
server_key_iv: "c4c2ccc834a25c73c876d2e40f6c5142"
server_key_version: "encryptionkey@0"
I20260925 07:25:29.551419 10162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/ts-2/wal/instance:
uuid: "0d316d7f90fd42c887a931fae5fd2f42"
format_stamp: "Formatted at 2026-09-25 07:25:29 on dist-test-slave-1drq"
server_key: "234fb9f608e4defbd803e549df009069"
server_key_iv: "c4c2ccc834a25c73c876d2e40f6c5142"
server_key_version: "encryptionkey@0"
I20260925 07:25:29.552646 10162 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 07:25:29.553258 10176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:29.553407 10162 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:29.553481 10162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/ts-2/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/ts-2/wal
uuid: "0d316d7f90fd42c887a931fae5fd2f42"
format_stamp: "Formatted at 2026-09-25 07:25:29 on dist-test-slave-1drq"
server_key: "234fb9f608e4defbd803e549df009069"
server_key_iv: "c4c2ccc834a25c73c876d2e40f6c5142"
server_key_version: "encryptionkey@0"
I20260925 07:25:29.553541 10162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:25:29.568260 10162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:25:29.569957 10162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:25:29.570071 10162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:25:29.570259 10162 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:25:29.570506 10162 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:25:29.570578 10162 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:29.570619 10162 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:25:29.570645 10162 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:29.575896 10162 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.67:45929
I20260925 07:25:29.575932 10289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.67:45929 every 8 connection(s)
I20260925 07:25:29.576160 10162 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
I20260925 07:25:29.579995 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 10162
I20260925 07:25:29.580120 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/ts-2/wal/instance
I20260925 07:25:29.580319 5753 external_mini_cluster.cc:1428] Setting key 096593dc22cef4d1f229cf63f52aba43
Sep 25 07:25:29 dist-test-slave-1drq krb5kdc[9790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321129, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.67@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:25:29.584105 9893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:29.576758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:35921 (local address 127.5.158.126:42707)
0925 07:25:29.576798 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:29.576800 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:29.576952 (+ 152us) server_negotiation.cc:407] Connection header received
0925 07:25:29.577410 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:29.577411 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:29.577441 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:29.577489 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:29.578264 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:29.578761 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:29.579597 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:29.579757 (+ 160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:29.579801 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:29.581917 (+ 2116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:29.581918 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:29.581919 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:29.581925 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:29.583210 (+ 1285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:29.583498 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:29.583499 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:29.583499 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:29.583541 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:29.583707 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:29.583707 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:29.583708 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:29.583790 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:29.583837 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:29.584001 (+ 164us) server_negotiation.cc:299] Negotiation successful
0925 07:25:29.584050 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 07:25:29.584241 10292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:29.576750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42707 (local address 127.5.158.67:35921)
0925 07:25:29.576860 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:29.576889 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:29.577317 (+ 428us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:29.577536 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:29.577539 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:29.577761 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:29.578195 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:29.578199 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:29.578825 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:29.578825 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:29.579526 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:29.579528 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:29.579815 (+ 287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:29.579815 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:29.579898 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:29.580311 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:29.580317 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:29.581842 (+ 1525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:29.583269 (+ 1427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:29.583273 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:29.583274 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:29.583437 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:29.583587 (+ 150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:29.583588 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:29.583588 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:29.583663 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:29.583835 (+ 172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:29.583839 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:29.583924 (+ 85us) client_negotiation.cc:769] Sending connection context
0925 07:25:29.583989 (+ 65us) client_negotiation.cc:240] Negotiation successful
0925 07:25:29.584083 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80,"thread_start_us":58,"threads_started":1}
I20260925 07:25:29.584635 10290 heartbeater.cc:347] Connected to a master server at 127.5.158.126:42707
I20260925 07:25:29.584723 10290 heartbeater.cc:464] Registering TS with master...
I20260925 07:25:29.584848 10290 heartbeater.cc:511] Master 127.5.158.126:42707 requested a full tablet report, sending...
I20260925 07:25:29.585206 9833 ts_manager.cc:194] Registered new tserver with Master: 0d316d7f90fd42c887a931fae5fd2f42 (127.5.158.67:45929)
I20260925 07:25:29.585539 9833 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.67@KRBTEST.COM'} at 127.5.158.67:35921
I20260925 07:25:29.592032 5753 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 07:25:29.598093 9893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:29.593172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35584 (local address 127.5.158.126:42707)
0925 07:25:29.593234 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:29.593235 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:29.593327 (+ 92us) server_negotiation.cc:407] Connection header received
0925 07:25:29.593362 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:29.593363 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:29.593369 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:29.593428 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:29.593919 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:29.594325 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:29.594912 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:29.595059 (+ 147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:29.595100 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:29.595987 (+ 887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:29.595988 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:29.595988 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:29.595993 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:29.597222 (+ 1229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:29.597507 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:29.597508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:29.597508 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:29.597552 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:29.597675 (+ 123us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:29.597676 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:29.597676 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:29.597788 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:29.597857 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:29.597936 (+ 79us) server_negotiation.cc:299] Negotiation successful
0925 07:25:29.597985 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260925 07:25:29.599354 9833 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35584:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 07:25:29.600666 9833 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 07:25:29.605973 10298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:29.602423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.66:34095 (local address 127.0.0.1:59450)
0925 07:25:29.602740 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:29.602774 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:29.602846 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:29.603041 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:29.603043 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:29.603045 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:25:29.603100 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:29.603102 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:29.604265 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:29.604266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:29.605005 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:29.605007 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:29.605710 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:29.605711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:29.605785 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:29.605797 (+ 12us) client_negotiation.cc:769] Sending connection context
0925 07:25:29.605867 (+ 70us) client_negotiation.cc:240] Negotiation successful
0925 07:25:29.605909 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":67,"threads_started":1}
I20260925 07:25:29.606227 10301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:29.602552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59450 (local address 127.5.158.66:34095)
0925 07:25:29.602705 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:29.602709 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:29.602844 (+ 135us) server_negotiation.cc:407] Connection header received
0925 07:25:29.602933 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:29.602935 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:29.602951 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:29.603025 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:25:29.603158 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:29.604169 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:29.605109 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:29.605613 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:29.605699 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:29.605753 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:29.605947 (+ 194us) server_negotiation.cc:299] Negotiation successful
0925 07:25:29.606006 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":64,"threads_started":1}
I20260925 07:25:29.606817 10299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:29.602631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.67:45929 (local address 127.0.0.1:44762)
0925 07:25:29.603011 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:29.603019 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:29.603062 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:29.603286 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:29.603287 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:29.603289 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:25:29.603323 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:29.603324 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:29.604104 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:29.604105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:29.605433 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:29.605435 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:29.606479 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:29.606480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:29.606582 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:29.606586 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 07:25:29.606638 (+ 52us) client_negotiation.cc:240] Negotiation successful
0925 07:25:29.606684 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"mutex_wait_us":24,"thread_start_us":57,"threads_started":1}
I20260925 07:25:29.606837 10303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:29.602731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44762 (local address 127.5.158.67:45929)
0925 07:25:29.602879 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:29.602895 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:29.603112 (+ 217us) server_negotiation.cc:407] Connection header received
0925 07:25:29.603146 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:29.603148 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:29.603162 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:29.603223 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:25:29.603390 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:29.603995 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:29.605510 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:29.606273 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:29.606404 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:29.606456 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:29.606737 (+ 281us) server_negotiation.cc:299] Negotiation successful
0925 07:25:29.606771 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":81,"threads_started":1}
I20260925 07:25:29.606822 10302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:29.602430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:44525 (local address 127.0.0.1:46770)
0925 07:25:29.602875 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:29.602883 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:29.602931 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:29.603188 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:29.603189 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:29.603191 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:25:29.603230 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:29.603231 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:29.604436 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:29.604437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:29.605434 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:29.605435 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:29.606473 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:29.606474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:29.606543 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:29.606564 (+ 21us) client_negotiation.cc:769] Sending connection context
0925 07:25:29.606643 (+ 79us) client_negotiation.cc:240] Negotiation successful
0925 07:25:29.606695 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"mutex_wait_us":8,"thread_start_us":89,"threads_started":1}
I20260925 07:25:29.607023 10300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:29.602535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46770 (local address 127.5.158.65:44525)
0925 07:25:29.602688 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:29.602692 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:29.602927 (+ 235us) server_negotiation.cc:407] Connection header received
0925 07:25:29.603002 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:29.603005 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:29.603019 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:29.603081 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:25:29.603303 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:29.604349 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:29.605506 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:29.606299 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:29.606377 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:29.606436 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:29.606874 (+ 438us) server_negotiation.cc:299] Negotiation successful
0925 07:25:29.606923 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":79,"threads_started":1}
I20260925 07:25:29.606951 10091 tablet_service.cc:1467] Processing CreateTablet for tablet 0837c4a8e01443e89c050bc476b6d7db (DEFAULT_TABLE table=test-table [id=b14a9fde736145e8af7fa083372a9a4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:25:29.607445 10091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0837c4a8e01443e89c050bc476b6d7db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:25:29.607709 9958 tablet_service.cc:1467] Processing CreateTablet for tablet 0837c4a8e01443e89c050bc476b6d7db (DEFAULT_TABLE table=test-table [id=b14a9fde736145e8af7fa083372a9a4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:25:29.607978 9958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0837c4a8e01443e89c050bc476b6d7db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:25:29.610438 10304 tablet_bootstrap.cc:492] T 0837c4a8e01443e89c050bc476b6d7db P 894bfc7462de40199e1cfa8f202e3f05: Bootstrap starting.
I20260925 07:25:29.610790 10305 tablet_bootstrap.cc:492] T 0837c4a8e01443e89c050bc476b6d7db P 3cd60fa02f094b3a94ebc217a1ec6b3b: Bootstrap starting.
I20260925 07:25:29.611109 10304 tablet_bootstrap.cc:654] T 0837c4a8e01443e89c050bc476b6d7db P 894bfc7462de40199e1cfa8f202e3f05: Neither blocks nor log segments found. Creating new log.
I20260925 07:25:29.611248 10224 tablet_service.cc:1467] Processing CreateTablet for tablet 0837c4a8e01443e89c050bc476b6d7db (DEFAULT_TABLE table=test-table [id=b14a9fde736145e8af7fa083372a9a4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:25:29.611384 10305 tablet_bootstrap.cc:654] T 0837c4a8e01443e89c050bc476b6d7db P 3cd60fa02f094b3a94ebc217a1ec6b3b: Neither blocks nor log segments found. Creating new log.
I20260925 07:25:29.611606 10224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0837c4a8e01443e89c050bc476b6d7db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:25:29.611785 10304 log.cc:824] T 0837c4a8e01443e89c050bc476b6d7db P 894bfc7462de40199e1cfa8f202e3f05: Log is configured to *not* fsync() on all Append() calls
I20260925 07:25:29.611941 10305 log.cc:824] T 0837c4a8e01443e89c050bc476b6d7db P 3cd60fa02f094b3a94ebc217a1ec6b3b: Log is configured to *not* fsync() on all Append() calls
I20260925 07:25:29.612545 10304 tablet_bootstrap.cc:492] T 0837c4a8e01443e89c050bc476b6d7db P 894bfc7462de40199e1cfa8f202e3f05: No bootstrap required, opened a new log
I20260925 07:25:29.612635 10304 ts_tablet_manager.cc:1397] T 0837c4a8e01443e89c050bc476b6d7db P 894bfc7462de40199e1cfa8f202e3f05: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 07:25:29.612804 10305 tablet_bootstrap.cc:492] T 0837c4a8e01443e89c050bc476b6d7db P 3cd60fa02f094b3a94ebc217a1ec6b3b: No bootstrap required, opened a new log
I20260925 07:25:29.612876 10305 ts_tablet_manager.cc:1397] T 0837c4a8e01443e89c050bc476b6d7db P 3cd60fa02f094b3a94ebc217a1ec6b3b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 07:25:29.614327 10308 tablet_bootstrap.cc:492] T 0837c4a8e01443e89c050bc476b6d7db P 0d316d7f90fd42c887a931fae5fd2f42: Bootstrap starting.
I20260925 07:25:29.614473 10304 raft_consensus.cc:348] T 0837c4a8e01443e89c050bc476b6d7db P 894bfc7462de40199e1cfa8f202e3f05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cd60fa02f094b3a94ebc217a1ec6b3b" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 44525 } } peers { permanent_uuid: "894bfc7462de40199e1cfa8f202e3f05" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 34095 } } peers { permanent_uuid: "0d316d7f90fd42c887a931fae5fd2f42" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 45929 } }
I20260925 07:25:29.614637 10304 raft_consensus.cc:374] T 0837c4a8e01443e89c050bc476b6d7db P 894bfc7462de40199e1cfa8f202e3f05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:25:29.614674 10304 raft_consensus.cc:729] T 0837c4a8e01443e89c050bc476b6d7db P 894bfc7462de40199e1cfa8f202e3f05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 894bfc7462de40199e1cfa8f202e3f05, State: Initialized, Role: FOLLOWER
I20260925 07:25:29.614657 10305 raft_consensus.cc:348] T 0837c4a8e01443e89c050bc476b6d7db P 3cd60fa02f094b3a94ebc217a1ec6b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cd60fa02f094b3a94ebc217a1ec6b3b" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 44525 } } peers { permanent_uuid: "894bfc7462de40199e1cfa8f202e3f05" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 34095 } } peers { permanent_uuid: "0d316d7f90fd42c887a931fae5fd2f42" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 45929 } }
I20260925 07:25:29.614777 10305 raft_consensus.cc:374] T 0837c4a8e01443e89c050bc476b6d7db P 3cd60fa02f094b3a94ebc217a1ec6b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:25:29.614810 10305 raft_consensus.cc:729] T 0837c4a8e01443e89c050bc476b6d7db P 3cd60fa02f094b3a94ebc217a1ec6b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cd60fa02f094b3a94ebc217a1ec6b3b, State: Initialized, Role: FOLLOWER
I20260925 07:25:29.614775 10304 consensus_queue.cc:260] T 0837c4a8e01443e89c050bc476b6d7db P 894bfc7462de40199e1cfa8f202e3f05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cd60fa02f094b3a94ebc217a1ec6b3b" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 44525 } } peers { permanent_uuid: "894bfc7462de40199e1cfa8f202e3f05" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 34095 } } peers { permanent_uuid: "0d316d7f90fd42c887a931fae5fd2f42" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 45929 } }
I20260925 07:25:29.614938 10305 consensus_queue.cc:260] T 0837c4a8e01443e89c050bc476b6d7db P 3cd60fa02f094b3a94ebc217a1ec6b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cd60fa02f094b3a94ebc217a1ec6b3b" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 44525 } } peers { permanent_uuid: "894bfc7462de40199e1cfa8f202e3f05" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 34095 } } peers { permanent_uuid: "0d316d7f90fd42c887a931fae5fd2f42" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 45929 } }
I20260925 07:25:29.615010 10308 tablet_bootstrap.cc:654] T 0837c4a8e01443e89c050bc476b6d7db P 0d316d7f90fd42c887a931fae5fd2f42: Neither blocks nor log segments found. Creating new log.
I20260925 07:25:29.615021 10304 ts_tablet_manager.cc:1428] T 0837c4a8e01443e89c050bc476b6d7db P 894bfc7462de40199e1cfa8f202e3f05: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 07:25:29.615088 10157 heartbeater.cc:503] Master 127.5.158.126:42707 was elected leader, sending a full tablet report...
I20260925 07:25:29.615185 10305 ts_tablet_manager.cc:1428] T 0837c4a8e01443e89c050bc476b6d7db P 3cd60fa02f094b3a94ebc217a1ec6b3b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 07:25:29.615283 10024 heartbeater.cc:503] Master 127.5.158.126:42707 was elected leader, sending a full tablet report...
I20260925 07:25:29.615567 10308 log.cc:824] T 0837c4a8e01443e89c050bc476b6d7db P 0d316d7f90fd42c887a931fae5fd2f42: Log is configured to *not* fsync() on all Append() calls
I20260925 07:25:29.616307 10308 tablet_bootstrap.cc:492] T 0837c4a8e01443e89c050bc476b6d7db P 0d316d7f90fd42c887a931fae5fd2f42: No bootstrap required, opened a new log
I20260925 07:25:29.616415 10308 ts_tablet_manager.cc:1397] T 0837c4a8e01443e89c050bc476b6d7db P 0d316d7f90fd42c887a931fae5fd2f42: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 07:25:29.617807 10308 raft_consensus.cc:348] T 0837c4a8e01443e89c050bc476b6d7db P 0d316d7f90fd42c887a931fae5fd2f42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cd60fa02f094b3a94ebc217a1ec6b3b" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 44525 } } peers { permanent_uuid: "894bfc7462de40199e1cfa8f202e3f05" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 34095 } } peers { permanent_uuid: "0d316d7f90fd42c887a931fae5fd2f42" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 45929 } }
I20260925 07:25:29.617949 10308 raft_consensus.cc:374] T 0837c4a8e01443e89c050bc476b6d7db P 0d316d7f90fd42c887a931fae5fd2f42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:25:29.617995 10308 raft_consensus.cc:729] T 0837c4a8e01443e89c050bc476b6d7db P 0d316d7f90fd42c887a931fae5fd2f42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d316d7f90fd42c887a931fae5fd2f42, State: Initialized, Role: FOLLOWER
I20260925 07:25:29.618108 10308 consensus_queue.cc:260] T 0837c4a8e01443e89c050bc476b6d7db P 0d316d7f90fd42c887a931fae5fd2f42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cd60fa02f094b3a94ebc217a1ec6b3b" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 44525 } } peers { permanent_uuid: "894bfc7462de40199e1cfa8f202e3f05" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 34095 } } peers { permanent_uuid: "0d316d7f90fd42c887a931fae5fd2f42" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 45929 } }
I20260925 07:25:29.618376 10308 ts_tablet_manager.cc:1428] T 0837c4a8e01443e89c050bc476b6d7db P 0d316d7f90fd42c887a931fae5fd2f42: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 07:25:29.618405 10290 heartbeater.cc:503] Master 127.5.158.126:42707 was elected leader, sending a full tablet report...
I20260925 07:25:29.634473 10312 raft_consensus.cc:482] T 0837c4a8e01443e89c050bc476b6d7db P 0d316d7f90fd42c887a931fae5fd2f42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 07:25:29.634541 10312 raft_consensus.cc:504] T 0837c4a8e01443e89c050bc476b6d7db P 0d316d7f90fd42c887a931fae5fd2f42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cd60fa02f094b3a94ebc217a1ec6b3b" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 44525 } } peers { permanent_uuid: "894bfc7462de40199e1cfa8f202e3f05" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 34095 } } peers { permanent_uuid: "0d316d7f90fd42c887a931fae5fd2f42" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 45929 } }
I20260925 07:25:29.634752 10312 leader_election.cc:288] T 0837c4a8e01443e89c050bc476b6d7db P 0d316d7f90fd42c887a931fae5fd2f42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3cd60fa02f094b3a94ebc217a1ec6b3b (127.5.158.65:44525), 894bfc7462de40199e1cfa8f202e3f05 (127.5.158.66:34095)
I20260925 07:25:29.637112 10292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:29.634866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:44525 (local address 127.5.158.67:53239)
0925 07:25:29.634896 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:29.634905 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:29.634939 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:29.635083 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:29.635084 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:29.635085 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:25:29.635152 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:29.635153 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:29.635691 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:29.635692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:29.636361 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:29.636363 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:29.636861 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:29.636861 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:29.636930 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:29.636932 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 07:25:29.636997 (+ 65us) client_negotiation.cc:240] Negotiation successful
0925 07:25:29.637032 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260925 07:25:29.637118 10300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:29.634919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:53239 (local address 127.5.158.65:44525)
0925 07:25:29.634953 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:29.634955 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:29.634963 (+ 8us) server_negotiation.cc:407] Connection header received
0925 07:25:29.634984 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:29.634985 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:29.634992 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:29.635077 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:25:29.635218 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:29.635621 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:29.636455 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:29.636779 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:29.636860 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:29.636867 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:29.637004 (+ 137us) server_negotiation.cc:299] Negotiation successful
0925 07:25:29.637036 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 07:25:29.637259 10301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:29.634940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:40429 (local address 127.5.158.66:34095)
0925 07:25:29.634973 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:29.634974 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:29.635073 (+ 99us) server_negotiation.cc:407] Connection header received
0925 07:25:29.635140 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:29.635141 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:29.635150 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:29.635204 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:25:29.635334 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:29.635725 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:29.636593 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:29.636940 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:29.637014 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:29.637020 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:29.637173 (+ 153us) server_negotiation.cc:299] Negotiation successful
0925 07:25:29.637202 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260925 07:25:29.637279 10313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:29.634866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.66:34095 (local address 127.5.158.67:40429)
0925 07:25:29.635017 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:29.635024 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:29.635080 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:29.635209 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:29.635210 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:29.635211 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:25:29.635269 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:29.635270 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:29.635796 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:29.635797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:29.636512 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:29.636513 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:29.637013 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:29.637014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:29.637114 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:29.637119 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 07:25:29.637169 (+ 50us) client_negotiation.cc:240] Negotiation successful
0925 07:25:29.637202 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":42,"thread_start_us":65,"threads_started":1}
I20260925 07:25:29.637547 9978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0837c4a8e01443e89c050bc476b6d7db" candidate_uuid: "0d316d7f90fd42c887a931fae5fd2f42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3cd60fa02f094b3a94ebc217a1ec6b3b" is_pre_election: true
I20260925 07:25:29.637637 10111 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0837c4a8e01443e89c050bc476b6d7db" candidate_uuid: "0d316d7f90fd42c887a931fae5fd2f42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "894bfc7462de40199e1cfa8f202e3f05" is_pre_election: true
I20260925 07:25:29.637670 9978 raft_consensus.cc:2454] T 0837c4a8e01443e89c050bc476b6d7db P 3cd60fa02f094b3a94ebc217a1ec6b3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d316d7f90fd42c887a931fae5fd2f42 in term 0.
I20260925 07:25:29.637738 10111 raft_consensus.cc:2454] T 0837c4a8e01443e89c050bc476b6d7db P 894bfc7462de40199e1cfa8f202e3f05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d316d7f90fd42c887a931fae5fd2f42 in term 0.
I20260925 07:25:29.637882 10180 leader_election.cc:302] T 0837c4a8e01443e89c050bc476b6d7db P 0d316d7f90fd42c887a931fae5fd2f42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d316d7f90fd42c887a931fae5fd2f42, 3cd60fa02f094b3a94ebc217a1ec6b3b; no voters:
I20260925 07:25:29.638015 10312 raft_consensus.cc:2781] T 0837c4a8e01443e89c050bc476b6d7db P 0d316d7f90fd42c887a931fae5fd2f42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 07:25:29.638093 10312 raft_consensus.cc:482] T 0837c4a8e01443e89c050bc476b6d7db P 0d316d7f90fd42c887a931fae5fd2f42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 07:25:29.638128 10312 raft_consensus.cc:3037] T 0837c4a8e01443e89c050bc476b6d7db P 0d316d7f90fd42c887a931fae5fd2f42 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:25:29.638851 10312 raft_consensus.cc:504] T 0837c4a8e01443e89c050bc476b6d7db P 0d316d7f90fd42c887a931fae5fd2f42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cd60fa02f094b3a94ebc217a1ec6b3b" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 44525 } } peers { permanent_uuid: "894bfc7462de40199e1cfa8f202e3f05" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 34095 } } peers { permanent_uuid: "0d316d7f90fd42c887a931fae5fd2f42" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 45929 } }
I20260925 07:25:29.638964 10312 leader_election.cc:288] T 0837c4a8e01443e89c050bc476b6d7db P 0d316d7f90fd42c887a931fae5fd2f42 [CANDIDATE]: Term 1 election: Requested vote from peers 3cd60fa02f094b3a94ebc217a1ec6b3b (127.5.158.65:44525), 894bfc7462de40199e1cfa8f202e3f05 (127.5.158.66:34095)
I20260925 07:25:29.639081 9978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0837c4a8e01443e89c050bc476b6d7db" candidate_uuid: "0d316d7f90fd42c887a931fae5fd2f42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3cd60fa02f094b3a94ebc217a1ec6b3b"
I20260925 07:25:29.639104 10111 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0837c4a8e01443e89c050bc476b6d7db" candidate_uuid: "0d316d7f90fd42c887a931fae5fd2f42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "894bfc7462de40199e1cfa8f202e3f05"
I20260925 07:25:29.639144 9978 raft_consensus.cc:3037] T 0837c4a8e01443e89c050bc476b6d7db P 3cd60fa02f094b3a94ebc217a1ec6b3b [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:25:29.639150 10111 raft_consensus.cc:3037] T 0837c4a8e01443e89c050bc476b6d7db P 894bfc7462de40199e1cfa8f202e3f05 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:25:29.639868 10111 raft_consensus.cc:2454] T 0837c4a8e01443e89c050bc476b6d7db P 894bfc7462de40199e1cfa8f202e3f05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d316d7f90fd42c887a931fae5fd2f42 in term 1.
I20260925 07:25:29.639899 9978 raft_consensus.cc:2454] T 0837c4a8e01443e89c050bc476b6d7db P 3cd60fa02f094b3a94ebc217a1ec6b3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d316d7f90fd42c887a931fae5fd2f42 in term 1.
I20260925 07:25:29.640057 10177 leader_election.cc:302] T 0837c4a8e01443e89c050bc476b6d7db P 0d316d7f90fd42c887a931fae5fd2f42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d316d7f90fd42c887a931fae5fd2f42, 894bfc7462de40199e1cfa8f202e3f05; no voters:
I20260925 07:25:29.640156 10312 raft_consensus.cc:2781] T 0837c4a8e01443e89c050bc476b6d7db P 0d316d7f90fd42c887a931fae5fd2f42 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:25:29.640269 10312 raft_consensus.cc:686] T 0837c4a8e01443e89c050bc476b6d7db P 0d316d7f90fd42c887a931fae5fd2f42 [term 1 LEADER]: Becoming Leader. State: Replica: 0d316d7f90fd42c887a931fae5fd2f42, State: Running, Role: LEADER
I20260925 07:25:29.640393 10312 consensus_queue.cc:237] T 0837c4a8e01443e89c050bc476b6d7db P 0d316d7f90fd42c887a931fae5fd2f42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cd60fa02f094b3a94ebc217a1ec6b3b" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 44525 } } peers { permanent_uuid: "894bfc7462de40199e1cfa8f202e3f05" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 34095 } } peers { permanent_uuid: "0d316d7f90fd42c887a931fae5fd2f42" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 45929 } }
I20260925 07:25:29.640978 9834 catalog_manager.cc:5495] T 0837c4a8e01443e89c050bc476b6d7db P 0d316d7f90fd42c887a931fae5fd2f42 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d316d7f90fd42c887a931fae5fd2f42 (127.5.158.67). New cstate: current_term: 1 leader_uuid: "0d316d7f90fd42c887a931fae5fd2f42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cd60fa02f094b3a94ebc217a1ec6b3b" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 44525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "894bfc7462de40199e1cfa8f202e3f05" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 34095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d316d7f90fd42c887a931fae5fd2f42" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 45929 } health_report { overall_health: HEALTHY } } }
I20260925 07:25:29.651742 10303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:29.649701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44766 (local address 127.5.158.67:45929)
0925 07:25:29.649730 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:29.649731 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:29.649739 (+ 8us) server_negotiation.cc:407] Connection header received
0925 07:25:29.649818 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:29.649819 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:29.649832 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:29.649890 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 07:25:29.649998 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:29.650381 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:29.651038 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:29.651220 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:29.651289 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:29.651432 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 07:25:29.651522 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 07:25:29.651595 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:29.651631 (+ 36us) server_negotiation.cc:299] Negotiation successful
0925 07:25:29.651647 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260925 07:25:29.653041 9978 raft_consensus.cc:1261] T 0837c4a8e01443e89c050bc476b6d7db P 3cd60fa02f094b3a94ebc217a1ec6b3b [term 1 FOLLOWER]: Refusing update from remote peer 0d316d7f90fd42c887a931fae5fd2f42: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:25:29.653046 10111 raft_consensus.cc:1261] T 0837c4a8e01443e89c050bc476b6d7db P 894bfc7462de40199e1cfa8f202e3f05 [term 1 FOLLOWER]: Refusing update from remote peer 0d316d7f90fd42c887a931fae5fd2f42: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:25:29.653302 10312 consensus_queue.cc:1035] T 0837c4a8e01443e89c050bc476b6d7db P 0d316d7f90fd42c887a931fae5fd2f42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "894bfc7462de40199e1cfa8f202e3f05" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 34095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:25:29.653430 10314 consensus_queue.cc:1035] T 0837c4a8e01443e89c050bc476b6d7db P 0d316d7f90fd42c887a931fae5fd2f42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3cd60fa02f094b3a94ebc217a1ec6b3b" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 44525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:25:29.655556 10317 mvcc.cc:205] Tried to move back new op lower bound from 7333155347057086464 to 7333155347007680512. Current Snapshot: MvccSnapshot[applied={T|T < 7333155347057086464}]
I20260925 07:25:29.657464 9834 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35584:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 07:25:29.657554 9834 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35584:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 07:25:29.658228 9834 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6a2a302f58dc4b059e7dd6c2dfb81057: Sending DeleteTablet for 3 replicas of tablet 0837c4a8e01443e89c050bc476b6d7db
I20260925 07:25:29.658613 10224 tablet_service.cc:1514] Processing DeleteTablet for tablet 0837c4a8e01443e89c050bc476b6d7db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:25:29 UTC) from {username='kudu', principal='kudu/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:44762
I20260925 07:25:29.658617 10091 tablet_service.cc:1514] Processing DeleteTablet for tablet 0837c4a8e01443e89c050bc476b6d7db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:25:29 UTC) from {username='kudu', principal='kudu/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:59450
I20260925 07:25:29.658917 10327 tablet_replica.cc:330] T 0837c4a8e01443e89c050bc476b6d7db P 894bfc7462de40199e1cfa8f202e3f05: stopping tablet replica
I20260925 07:25:29.658941 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 9896
I20260925 07:25:29.658929 9958 tablet_service.cc:1514] Processing DeleteTablet for tablet 0837c4a8e01443e89c050bc476b6d7db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:25:29 UTC) from {username='kudu', principal='kudu/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:46770
I20260925 07:25:29.659412 10327 raft_consensus.cc:2229] T 0837c4a8e01443e89c050bc476b6d7db P 894bfc7462de40199e1cfa8f202e3f05 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 07:25:29.659528 10327 raft_consensus.cc:2258] T 0837c4a8e01443e89c050bc476b6d7db P 894bfc7462de40199e1cfa8f202e3f05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:25:29.659952 10327 ts_tablet_manager.cc:1905] T 0837c4a8e01443e89c050bc476b6d7db P 894bfc7462de40199e1cfa8f202e3f05: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:25:29.661360 10327 ts_tablet_manager.cc:1918] T 0837c4a8e01443e89c050bc476b6d7db P 894bfc7462de40199e1cfa8f202e3f05: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 07:25:29.661425 10327 log.cc:1196] T 0837c4a8e01443e89c050bc476b6d7db P 894bfc7462de40199e1cfa8f202e3f05: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790321110323796-5753-0/minicluster-data/ts-1/wal/wals/0837c4a8e01443e89c050bc476b6d7db
I20260925 07:25:29.661677 10327 ts_tablet_manager.cc:1939] T 0837c4a8e01443e89c050bc476b6d7db P 894bfc7462de40199e1cfa8f202e3f05: Deleting consensus metadata
I20260925 07:25:29.662070 9818 catalog_manager.cc:4841] TS 894bfc7462de40199e1cfa8f202e3f05 (127.5.158.66:34095): tablet 0837c4a8e01443e89c050bc476b6d7db (table test-table [id=b14a9fde736145e8af7fa083372a9a4a]) successfully deleted
W20260925 07:25:29.664734 9821 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.158.65:44525 (error 108)
I20260925 07:25:29.664923 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 10029
I20260925 07:25:29.665073 10328 tablet_replica.cc:330] T 0837c4a8e01443e89c050bc476b6d7db P 0d316d7f90fd42c887a931fae5fd2f42: stopping tablet replica
I20260925 07:25:29.665172 10328 raft_consensus.cc:2229] T 0837c4a8e01443e89c050bc476b6d7db P 0d316d7f90fd42c887a931fae5fd2f42 [term 1 LEADER]: Raft consensus shutting down.
I20260925 07:25:29.665244 10328 raft_consensus.cc:2258] T 0837c4a8e01443e89c050bc476b6d7db P 0d316d7f90fd42c887a931fae5fd2f42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:25:29.665603 10299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:29.664997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:44525 (local address 127.0.0.1:46772)
0925 07:25:29.665541 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:29.665578 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.65:44525: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":528}
W20260925 07:25:29.665704 9821 catalog_manager.cc:4584] TS 3cd60fa02f094b3a94ebc217a1ec6b3b (127.5.158.65:44525): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0837c4a8e01443e89c050bc476b6d7db: Network error: Client connection negotiation failed: client connection to 127.5.158.65:44525: connect: Connection refused (error 111)
I20260925 07:25:29.669267 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 10162
I20260925 07:25:29.673244 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 9803
I20260925 07:25:29.673255 10299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:29.673168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.67:45929 (local address 127.0.0.1:44768)
0925 07:25:29.673210 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:29.673237 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.67:45929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
2026-09-25T07:25:29Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (1706 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-25T07:25:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T07:25:29Z Disabled control of system clock
I20260925 07:25:29.699329 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:44811
--webserver_interface=127.5.158.126
--webserver_port=0
--builtin_ntp_servers=127.5.158.84:46855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.158.126:44811
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260925 07:25:29.770908 10333 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:25:29.771070 10333 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:25:29.771086 10333 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:25:29.771966 10333 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:25:29.771994 10333 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:25:29.772006 10333 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:25:29.772015 10333 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:25:29.772025 10333 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:25:29.773232 10333 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:46855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.158.126:44811
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:44811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10333
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:25:29.773438 10333 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:25:29.773725 10333 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:25:29.777966 10338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:29.777993 10341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:29.777942 10339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:29.778445 10333 server_base.cc:1034] running on GCE node
I20260925 07:25:29.778745 10333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:25:29.779006 10333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:25:29.780282 10333 hybrid_clock.cc:648] HybridClock initialized: now 1790321129780253 us; error 41 us; skew 500 ppm
I20260925 07:25:29.781855 10333 webserver.cc:460] Webserver started at http://127.5.158.126:45153/ using document root <none> and password file <none>
I20260925 07:25:29.782063 10333 fs_manager.cc:359] Metadata directory not provided
I20260925 07:25:29.782114 10333 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:25:29.782230 10333 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:25:29.783277 10333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/master-0/data/instance:
uuid: "acb46a295e51498d8f0a3a9bbe742184"
format_stamp: "Formatted at 2026-09-25 07:25:29 on dist-test-slave-1drq"
server_key: "5dfa2cbaee9b8bf639887632f098bc0e"
server_key_iv: "6ce98a60918a9c86bcbb56834f166ae9"
server_key_version: "encryptionkey@0"
I20260925 07:25:29.783602 10333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/master-0/wal/instance:
uuid: "acb46a295e51498d8f0a3a9bbe742184"
format_stamp: "Formatted at 2026-09-25 07:25:29 on dist-test-slave-1drq"
server_key: "5dfa2cbaee9b8bf639887632f098bc0e"
server_key_iv: "6ce98a60918a9c86bcbb56834f166ae9"
server_key_version: "encryptionkey@0"
I20260925 07:25:29.784750 10333 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 07:25:29.785415 10347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:29.785565 10333 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:29.785661 10333 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "acb46a295e51498d8f0a3a9bbe742184"
format_stamp: "Formatted at 2026-09-25 07:25:29 on dist-test-slave-1drq"
server_key: "5dfa2cbaee9b8bf639887632f098bc0e"
server_key_iv: "6ce98a60918a9c86bcbb56834f166ae9"
server_key_version: "encryptionkey@0"
I20260925 07:25:29.785732 10333 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:25:29.788715 10333 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:25:29.788931 10333 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:25:29.789047 10333 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:25:29.792476 10333 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.126:44811
I20260925 07:25:29.792553 10399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.126:44811 every 8 connection(s)
I20260925 07:25:29.792807 10333 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:25:29.793406 10400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:25:29.795678 10400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acb46a295e51498d8f0a3a9bbe742184: Bootstrap starting.
I20260925 07:25:29.796185 10400 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P acb46a295e51498d8f0a3a9bbe742184: Neither blocks nor log segments found. Creating new log.
I20260925 07:25:29.796406 10400 log.cc:824] T 00000000000000000000000000000000 P acb46a295e51498d8f0a3a9bbe742184: Log is configured to *not* fsync() on all Append() calls
I20260925 07:25:29.796984 10400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acb46a295e51498d8f0a3a9bbe742184: No bootstrap required, opened a new log
I20260925 07:25:29.798516 10400 raft_consensus.cc:348] T 00000000000000000000000000000000 P acb46a295e51498d8f0a3a9bbe742184 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acb46a295e51498d8f0a3a9bbe742184" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 44811 } }
I20260925 07:25:29.798671 10400 raft_consensus.cc:374] T 00000000000000000000000000000000 P acb46a295e51498d8f0a3a9bbe742184 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:25:29.798692 10400 raft_consensus.cc:729] T 00000000000000000000000000000000 P acb46a295e51498d8f0a3a9bbe742184 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acb46a295e51498d8f0a3a9bbe742184, State: Initialized, Role: FOLLOWER
I20260925 07:25:29.798821 10400 consensus_queue.cc:260] T 00000000000000000000000000000000 P acb46a295e51498d8f0a3a9bbe742184 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acb46a295e51498d8f0a3a9bbe742184" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 44811 } }
I20260925 07:25:29.798894 10400 raft_consensus.cc:388] T 00000000000000000000000000000000 P acb46a295e51498d8f0a3a9bbe742184 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 07:25:29.798916 10400 raft_consensus.cc:482] T 00000000000000000000000000000000 P acb46a295e51498d8f0a3a9bbe742184 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 07:25:29.798949 10400 raft_consensus.cc:3037] T 00000000000000000000000000000000 P acb46a295e51498d8f0a3a9bbe742184 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:25:29.799443 10400 raft_consensus.cc:504] T 00000000000000000000000000000000 P acb46a295e51498d8f0a3a9bbe742184 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acb46a295e51498d8f0a3a9bbe742184" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 44811 } }
I20260925 07:25:29.799520 10400 leader_election.cc:302] T 00000000000000000000000000000000 P acb46a295e51498d8f0a3a9bbe742184 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: acb46a295e51498d8f0a3a9bbe742184; no voters:
I20260925 07:25:29.799633 10400 leader_election.cc:288] T 00000000000000000000000000000000 P acb46a295e51498d8f0a3a9bbe742184 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 07:25:29.799693 10403 raft_consensus.cc:2781] T 00000000000000000000000000000000 P acb46a295e51498d8f0a3a9bbe742184 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:25:29.799836 10403 raft_consensus.cc:686] T 00000000000000000000000000000000 P acb46a295e51498d8f0a3a9bbe742184 [term 1 LEADER]: Becoming Leader. State: Replica: acb46a295e51498d8f0a3a9bbe742184, State: Running, Role: LEADER
I20260925 07:25:29.799861 10400 sys_catalog.cc:564] T 00000000000000000000000000000000 P acb46a295e51498d8f0a3a9bbe742184 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:25:29.799912 10403 consensus_queue.cc:237] T 00000000000000000000000000000000 P acb46a295e51498d8f0a3a9bbe742184 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acb46a295e51498d8f0a3a9bbe742184" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 44811 } }
I20260925 07:25:29.800256 10404 sys_catalog.cc:455] T 00000000000000000000000000000000 P acb46a295e51498d8f0a3a9bbe742184 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "acb46a295e51498d8f0a3a9bbe742184" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acb46a295e51498d8f0a3a9bbe742184" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 44811 } } }
I20260925 07:25:29.800341 10404 sys_catalog.cc:458] T 00000000000000000000000000000000 P acb46a295e51498d8f0a3a9bbe742184 [sys.catalog]: This master's current role is: LEADER
I20260925 07:25:29.800441 10405 sys_catalog.cc:455] T 00000000000000000000000000000000 P acb46a295e51498d8f0a3a9bbe742184 [sys.catalog]: SysCatalogTable state changed. Reason: New leader acb46a295e51498d8f0a3a9bbe742184. Latest consensus state: current_term: 1 leader_uuid: "acb46a295e51498d8f0a3a9bbe742184" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acb46a295e51498d8f0a3a9bbe742184" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 44811 } } }
I20260925 07:25:29.800503 10405 sys_catalog.cc:458] T 00000000000000000000000000000000 P acb46a295e51498d8f0a3a9bbe742184 [sys.catalog]: This master's current role is: LEADER
I20260925 07:25:29.801038 10410 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 07:25:29.801415 10410 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 07:25:29.802767 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 10333
I20260925 07:25:29.802840 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/master-0/wal/instance
I20260925 07:25:29.803007 5753 external_mini_cluster.cc:1428] Setting key 77d00690c4b1a1dc13a25c18dab29624
I20260925 07:25:29.803182 10410 catalog_manager.cc:1347] Generated new cluster ID: 80f2a409220144d6a9040430a5ce60c0
I20260925 07:25:29.803228 10410 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 07:25:29.805114 10423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:29.803543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37768 (local address 127.5.158.126:44811)
0925 07:25:29.803679 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:29.803686 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:29.803709 (+ 23us) server_negotiation.cc:407] Connection header received
0925 07:25:29.803803 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:29.803808 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:29.803848 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:29.803946 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:29.804693 (+ 747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:29.804696 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:29.804697 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:25:29.804700 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:29.804743 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:25:29.804755 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:29.804807 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:29.804895 (+ 88us) server_negotiation.cc:299] Negotiation successful
0925 07:25:29.804940 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":62,"threads_started":1}
I20260925 07:25:29.811347 10410 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 07:25:29.811465 10410 catalog_manager.cc:1504] Loading token signing keys...
I20260925 07:25:29.819238 10410 catalog_manager.cc:5868] T 00000000000000000000000000000000 P acb46a295e51498d8f0a3a9bbe742184: Generated new TSK 0
I20260925 07:25:29.819361 10410 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 07:25:29.822877 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.65:0
--local_ip_for_outbound_sockets=127.5.158.65
--webserver_interface=127.5.158.65
--webserver_port=0
--tserver_master_addrs=127.5.158.126:44811
--builtin_ntp_servers=127.5.158.84:46855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260925 07:25:29.893875 10424 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:25:29.894061 10424 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:25:29.894091 10424 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:25:29.895102 10424 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:25:29.895151 10424 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:25:29.895190 10424 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.65
I20260925 07:25:29.896359 10424 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:46855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.158.65
--webserver_port=0
--tserver_master_addrs=127.5.158.126:44811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10424
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.65
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:25:29.896585 10424 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:25:29.896824 10424 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:25:29.901327 10429 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:29.901451 10430 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:29.901324 10432 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:29.901661 10424 server_base.cc:1034] running on GCE node
I20260925 07:25:29.901841 10424 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:25:29.902086 10424 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:25:29.903242 10424 hybrid_clock.cc:648] HybridClock initialized: now 1790321129903223 us; error 32 us; skew 500 ppm
I20260925 07:25:29.904523 10424 webserver.cc:460] Webserver started at http://127.5.158.65:41737/ using document root <none> and password file <none>
I20260925 07:25:29.904767 10424 fs_manager.cc:359] Metadata directory not provided
I20260925 07:25:29.904819 10424 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:25:29.904954 10424 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:25:29.906220 10424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/ts-0/data/instance:
uuid: "3f2f6698dd594af1b4130cd54ef1d343"
format_stamp: "Formatted at 2026-09-25 07:25:29 on dist-test-slave-1drq"
server_key: "0ea861592102c47f63bd95e2d8a4ff5a"
server_key_iv: "735d33282c5ab489f018dc73a8c13633"
server_key_version: "encryptionkey@0"
I20260925 07:25:29.906618 10424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/ts-0/wal/instance:
uuid: "3f2f6698dd594af1b4130cd54ef1d343"
format_stamp: "Formatted at 2026-09-25 07:25:29 on dist-test-slave-1drq"
server_key: "0ea861592102c47f63bd95e2d8a4ff5a"
server_key_iv: "735d33282c5ab489f018dc73a8c13633"
server_key_version: "encryptionkey@0"
I20260925 07:25:29.908226 10424 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 07:25:29.909053 10438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:29.909231 10424 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:25:29.909315 10424 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/ts-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/ts-0/wal
uuid: "3f2f6698dd594af1b4130cd54ef1d343"
format_stamp: "Formatted at 2026-09-25 07:25:29 on dist-test-slave-1drq"
server_key: "0ea861592102c47f63bd95e2d8a4ff5a"
server_key_iv: "735d33282c5ab489f018dc73a8c13633"
server_key_version: "encryptionkey@0"
I20260925 07:25:29.909384 10424 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:25:29.912060 10424 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:25:29.912278 10424 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:25:29.912397 10424 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:25:29.912585 10424 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:25:29.912829 10424 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:25:29.912871 10424 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:29.912904 10424 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:25:29.912930 10424 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:29.918740 10424 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.65:43661
I20260925 07:25:29.918804 10551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.65:43661 every 8 connection(s)
I20260925 07:25:29.919047 10424 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
I20260925 07:25:29.921846 10423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:29.919694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:45989 (local address 127.5.158.126:44811)
0925 07:25:29.919756 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:29.919768 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:29.919954 (+ 186us) server_negotiation.cc:407] Connection header received
0925 07:25:29.920405 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:29.920406 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:29.920415 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:29.920478 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:29.921574 (+ 1096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:29.921575 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:29.921576 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:25:29.921581 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:29.921597 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:25:29.921602 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:29.921666 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:29.921760 (+ 94us) server_negotiation.cc:299] Negotiation successful
0925 07:25:29.921775 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 07:25:29.922008 10554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:29.919674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:44811 (local address 127.5.158.65:45989)
0925 07:25:29.919850 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:29.919878 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:29.920283 (+ 405us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:29.920536 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:29.920540 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:29.920548 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:29.921433 (+ 885us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 07:25:29.921434 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:29.921458 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 07:25:29.921483 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:29.921664 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:29.921669 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:29.921683 (+ 14us) client_negotiation.cc:769] Sending connection context
0925 07:25:29.921766 (+ 83us) client_negotiation.cc:240] Negotiation successful
0925 07:25:29.921819 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":73,"threads_started":1}
I20260925 07:25:29.922433 10552 heartbeater.cc:347] Connected to a master server at 127.5.158.126:44811
I20260925 07:25:29.922516 10552 heartbeater.cc:464] Registering TS with master...
I20260925 07:25:29.922646 10552 heartbeater.cc:511] Master 127.5.158.126:44811 requested a full tablet report, sending...
I20260925 07:25:29.923067 10364 ts_manager.cc:194] Registered new tserver with Master: 3f2f6698dd594af1b4130cd54ef1d343 (127.5.158.65:43661)
I20260925 07:25:29.926877 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 10424
I20260925 07:25:29.926985 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/ts-0/wal/instance
I20260925 07:25:29.927134 5753 external_mini_cluster.cc:1428] Setting key 24824b730b28ee554997bfc8f28ed570
I20260925 07:25:29.927991 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.66:0
--local_ip_for_outbound_sockets=127.5.158.66
--webserver_interface=127.5.158.66
--webserver_port=0
--tserver_master_addrs=127.5.158.126:44811
--builtin_ntp_servers=127.5.158.84:46855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260925 07:25:30.002956 10555 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:25:30.003151 10555 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:25:30.003172 10555 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:25:30.004139 10555 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:25:30.004174 10555 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:25:30.004199 10555 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.66
I20260925 07:25:30.005486 10555 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:46855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.158.66
--webserver_port=0
--tserver_master_addrs=127.5.158.126:44811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10555
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.66
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:25:30.005651 10555 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:25:30.005872 10555 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:25:30.010413 10561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:30.010392 10563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:30.010439 10555 server_base.cc:1034] running on GCE node
W20260925 07:25:30.010393 10560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:30.010769 10555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:25:30.010980 10555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:25:30.012143 10555 hybrid_clock.cc:648] HybridClock initialized: now 1790321130012112 us; error 27 us; skew 500 ppm
I20260925 07:25:30.013278 10555 webserver.cc:460] Webserver started at http://127.5.158.66:40639/ using document root <none> and password file <none>
I20260925 07:25:30.013484 10555 fs_manager.cc:359] Metadata directory not provided
I20260925 07:25:30.013525 10555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:25:30.013661 10555 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:25:30.014530 10555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/ts-1/data/instance:
uuid: "0c5e0a40f69c4c34ac68e46cf2cfb329"
format_stamp: "Formatted at 2026-09-25 07:25:30 on dist-test-slave-1drq"
server_key: "a7e779f70c9677a2cacf65a2b4a45c65"
server_key_iv: "107222778782826c8a30b63de07426b3"
server_key_version: "encryptionkey@0"
I20260925 07:25:30.014842 10555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/ts-1/wal/instance:
uuid: "0c5e0a40f69c4c34ac68e46cf2cfb329"
format_stamp: "Formatted at 2026-09-25 07:25:30 on dist-test-slave-1drq"
server_key: "a7e779f70c9677a2cacf65a2b4a45c65"
server_key_iv: "107222778782826c8a30b63de07426b3"
server_key_version: "encryptionkey@0"
I20260925 07:25:30.015976 10555 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 07:25:30.016607 10569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:30.016758 10555 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:25:30.016863 10555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/ts-1/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/ts-1/wal
uuid: "0c5e0a40f69c4c34ac68e46cf2cfb329"
format_stamp: "Formatted at 2026-09-25 07:25:30 on dist-test-slave-1drq"
server_key: "a7e779f70c9677a2cacf65a2b4a45c65"
server_key_iv: "107222778782826c8a30b63de07426b3"
server_key_version: "encryptionkey@0"
I20260925 07:25:30.016929 10555 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:25:30.020043 10555 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:25:30.020278 10555 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:25:30.020401 10555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:25:30.020598 10555 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:25:30.020874 10555 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:25:30.020922 10555 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:30.020959 10555 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:25:30.020987 10555 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:30.026299 10555 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.66:36769
I20260925 07:25:30.026335 10682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.66:36769 every 8 connection(s)
I20260925 07:25:30.026623 10555 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
I20260925 07:25:30.029114 10423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:30.027231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:58473 (local address 127.5.158.126:44811)
0925 07:25:30.027301 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:30.027304 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:30.027494 (+ 190us) server_negotiation.cc:407] Connection header received
0925 07:25:30.027868 (+ 374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:30.027869 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:30.027874 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:30.027920 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:30.028908 (+ 988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:30.028909 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:30.028910 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:25:30.028914 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:30.028928 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:25:30.028932 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:30.028977 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:30.029021 (+ 44us) server_negotiation.cc:299] Negotiation successful
0925 07:25:30.029033 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 07:25:30.029299 10685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:30.027197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:44811 (local address 127.5.158.66:58473)
0925 07:25:30.027361 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:30.027390 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:30.027794 (+ 404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:30.027973 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:30.027976 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:30.027983 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:30.028793 (+ 810us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 07:25:30.028793 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:30.028816 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 07:25:30.028840 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:30.028974 (+ 134us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:30.028976 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:30.028978 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 07:25:30.029026 (+ 48us) client_negotiation.cc:240] Negotiation successful
0925 07:25:30.029065 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":83,"threads_started":1}
I20260925 07:25:30.029969 10683 heartbeater.cc:347] Connected to a master server at 127.5.158.126:44811
I20260925 07:25:30.030063 10683 heartbeater.cc:464] Registering TS with master...
I20260925 07:25:30.030190 10683 heartbeater.cc:511] Master 127.5.158.126:44811 requested a full tablet report, sending...
I20260925 07:25:30.030545 10364 ts_manager.cc:194] Registered new tserver with Master: 0c5e0a40f69c4c34ac68e46cf2cfb329 (127.5.158.66:36769)
I20260925 07:25:30.032053 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 10555
I20260925 07:25:30.032132 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/ts-1/wal/instance
I20260925 07:25:30.032292 5753 external_mini_cluster.cc:1428] Setting key 8dcd53dd26bc5d88e0e54f889e8e764f
I20260925 07:25:30.033171 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.67:0
--local_ip_for_outbound_sockets=127.5.158.67
--webserver_interface=127.5.158.67
--webserver_port=0
--tserver_master_addrs=127.5.158.126:44811
--builtin_ntp_servers=127.5.158.84:46855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260925 07:25:30.109596 10686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:25:30.109787 10686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:25:30.109808 10686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:25:30.110769 10686 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:25:30.110822 10686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:25:30.110849 10686 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.67
I20260925 07:25:30.112143 10686 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:46855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.158.67
--webserver_port=0
--tserver_master_addrs=127.5.158.126:44811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10686
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.67
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:25:30.112336 10686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:25:30.112608 10686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:25:30.116907 10692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:30.116920 10694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:30.116920 10691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:30.117046 10686 server_base.cc:1034] running on GCE node
I20260925 07:25:30.117334 10686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:25:30.117583 10686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:25:30.118719 10686 hybrid_clock.cc:648] HybridClock initialized: now 1790321130118708 us; error 28 us; skew 500 ppm
I20260925 07:25:30.119886 10686 webserver.cc:460] Webserver started at http://127.5.158.67:42813/ using document root <none> and password file <none>
I20260925 07:25:30.120124 10686 fs_manager.cc:359] Metadata directory not provided
I20260925 07:25:30.120190 10686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:25:30.120326 10686 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:25:30.121251 10686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/ts-2/data/instance:
uuid: "fdae728231294bee91c49f05531682e0"
format_stamp: "Formatted at 2026-09-25 07:25:30 on dist-test-slave-1drq"
server_key: "8d0daf47166221d4134555bb94e9fa9f"
server_key_iv: "894d48d729395e9f643e0c4c634d47dd"
server_key_version: "encryptionkey@0"
I20260925 07:25:30.121524 10686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/ts-2/wal/instance:
uuid: "fdae728231294bee91c49f05531682e0"
format_stamp: "Formatted at 2026-09-25 07:25:30 on dist-test-slave-1drq"
server_key: "8d0daf47166221d4134555bb94e9fa9f"
server_key_iv: "894d48d729395e9f643e0c4c634d47dd"
server_key_version: "encryptionkey@0"
I20260925 07:25:30.122604 10686 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 07:25:30.123152 10700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:30.123301 10686 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 07:25:30.123399 10686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/ts-2/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/ts-2/wal
uuid: "fdae728231294bee91c49f05531682e0"
format_stamp: "Formatted at 2026-09-25 07:25:30 on dist-test-slave-1drq"
server_key: "8d0daf47166221d4134555bb94e9fa9f"
server_key_iv: "894d48d729395e9f643e0c4c634d47dd"
server_key_version: "encryptionkey@0"
I20260925 07:25:30.123474 10686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:25:30.126603 10686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:25:30.126832 10686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:25:30.126951 10686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:25:30.127138 10686 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:25:30.127435 10686 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:25:30.127466 10686 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:30.127512 10686 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:25:30.127539 10686 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:30.132603 10686 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.67:36889
I20260925 07:25:30.132647 10813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.67:36889 every 8 connection(s)
I20260925 07:25:30.132871 10686 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
I20260925 07:25:30.135181 10423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:30.133394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:37269 (local address 127.5.158.126:44811)
0925 07:25:30.133431 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:30.133433 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:30.133605 (+ 172us) server_negotiation.cc:407] Connection header received
0925 07:25:30.133973 (+ 368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:30.133974 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:30.133981 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:30.134029 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:30.134949 (+ 920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:30.134950 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:30.134951 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:25:30.134954 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:30.134969 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:25:30.134973 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:30.135026 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:30.135103 (+ 77us) server_negotiation.cc:299] Negotiation successful
0925 07:25:30.135117 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 07:25:30.135344 10816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:30.133390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:44811 (local address 127.5.158.67:37269)
0925 07:25:30.133517 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:30.133540 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:30.133886 (+ 346us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:30.134082 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:30.134085 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:30.134097 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:30.134828 (+ 731us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 07:25:30.134828 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:30.134848 (+ 20us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 07:25:30.134869 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:30.135026 (+ 157us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:30.135030 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:30.135033 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 07:25:30.135110 (+ 77us) client_negotiation.cc:240] Negotiation successful
0925 07:25:30.135159 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":65,"threads_started":1}
I20260925 07:25:30.135768 10814 heartbeater.cc:347] Connected to a master server at 127.5.158.126:44811
I20260925 07:25:30.135842 10814 heartbeater.cc:464] Registering TS with master...
I20260925 07:25:30.135984 10814 heartbeater.cc:511] Master 127.5.158.126:44811 requested a full tablet report, sending...
I20260925 07:25:30.136305 10364 ts_manager.cc:194] Registered new tserver with Master: fdae728231294bee91c49f05531682e0 (127.5.158.67:36889)
I20260925 07:25:30.137209 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 10686
I20260925 07:25:30.137287 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/ts-2/wal/instance
I20260925 07:25:30.137427 5753 external_mini_cluster.cc:1428] Setting key a727856d3c480bfe396f7f91bec3d0b5
I20260925 07:25:30.138186 5753 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 07:25:30.140179 10423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:30.139300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37774 (local address 127.5.158.126:44811)
0925 07:25:30.139332 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:30.139334 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:30.139438 (+ 104us) server_negotiation.cc:407] Connection header received
0925 07:25:30.139511 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:30.139512 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:30.139516 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:30.139571 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:30.140001 (+ 430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:30.140002 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:30.140003 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:25:30.140006 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:30.140017 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:25:30.140020 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:30.140065 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:30.140111 (+ 46us) server_negotiation.cc:299] Negotiation successful
0925 07:25:30.140125 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260925 07:25:30.141333 10364 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37774:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 07:25:30.142591 10364 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 07:25:30.145742 10825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:30.144284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.66:36769 (local address 127.0.0.1:51286)
0925 07:25:30.144684 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:30.144694 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:30.144751 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:30.144895 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:30.144900 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:30.144905 (+ 5us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:30.145379 (+ 474us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 07:25:30.145379 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:30.145392 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 07:25:30.145412 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:30.145597 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:30.145601 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:30.145603 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 07:25:30.145663 (+ 60us) client_negotiation.cc:240] Negotiation successful
0925 07:25:30.145675 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":383,"mutex_wait_us":1,"thread_start_us":57,"threads_started":1}
I20260925 07:25:30.145773 10827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:30.144590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51286 (local address 127.5.158.66:36769)
0925 07:25:30.144772 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:30.144774 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:30.144783 (+ 9us) server_negotiation.cc:407] Connection header received
0925 07:25:30.144803 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:30.144817 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:30.144826 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:30.144899 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:30.145481 (+ 582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:30.145487 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:30.145488 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:25:30.145491 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:30.145522 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:25:30.145549 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:30.145604 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:30.145670 (+ 66us) server_negotiation.cc:299] Negotiation successful
0925 07:25:30.145686 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":83,"threads_started":1}
I20260925 07:25:30.145886 10823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:30.144284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:43661 (local address 127.0.0.1:46132)
0925 07:25:30.144448 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:30.144485 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:30.144540 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:30.145004 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:30.145005 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:30.145007 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:30.145486 (+ 479us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 07:25:30.145487 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:30.145491 (+ 4us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 07:25:30.145498 (+ 7us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:30.145758 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:30.145759 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:30.145760 (+ 1us) client_negotiation.cc:769] Sending connection context
0925 07:25:30.145811 (+ 51us) client_negotiation.cc:240] Negotiation successful
0925 07:25:30.145823 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":86,"threads_started":1}
I20260925 07:25:30.146144 10824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:30.144411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46132 (local address 127.5.158.65:43661)
0925 07:25:30.144866 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:30.144871 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:30.144880 (+ 9us) server_negotiation.cc:407] Connection header received
0925 07:25:30.144903 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:30.144905 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:30.144913 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:30.145138 (+ 225us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:30.145666 (+ 528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:30.145670 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:30.145670 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:25:30.145673 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:30.145697 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:25:30.145710 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:30.145762 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:30.146032 (+ 270us) server_negotiation.cc:299] Negotiation successful
0925 07:25:30.146047 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":84,"threads_started":1}
I20260925 07:25:30.146240 10822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:30.144474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.67:36889 (local address 127.0.0.1:48498)
0925 07:25:30.144764 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:30.144773 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:30.144816 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:30.145126 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:30.145127 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:30.145129 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:30.145914 (+ 785us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 07:25:30.145914 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:30.145920 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 07:25:30.145929 (+ 9us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:30.146116 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:30.146117 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:30.146118 (+ 1us) client_negotiation.cc:769] Sending connection context
0925 07:25:30.146170 (+ 52us) client_negotiation.cc:240] Negotiation successful
0925 07:25:30.146182 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":54,"threads_started":1}
I20260925 07:25:30.146247 10826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:30.144543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48498 (local address 127.5.158.67:36889)
0925 07:25:30.144684 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:30.144688 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:30.144828 (+ 140us) server_negotiation.cc:407] Connection header received
0925 07:25:30.144863 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:30.144865 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:30.144872 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:30.144929 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:30.146002 (+ 1073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:30.146007 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:30.146008 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:25:30.146011 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:30.146039 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:25:30.146056 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:30.146107 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:30.146173 (+ 66us) server_negotiation.cc:299] Negotiation successful
0925 07:25:30.146185 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":72,"threads_started":1}
I20260925 07:25:30.146386 10617 tablet_service.cc:1467] Processing CreateTablet for tablet e242d90b5bdc4698a23a8017371e558b (DEFAULT_TABLE table=test-table [id=e761ad179e0f487688e56bc528cffa22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:25:30.146680 10617 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e242d90b5bdc4698a23a8017371e558b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:25:30.146682 10486 tablet_service.cc:1467] Processing CreateTablet for tablet e242d90b5bdc4698a23a8017371e558b (DEFAULT_TABLE table=test-table [id=e761ad179e0f487688e56bc528cffa22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:25:30.146772 10748 tablet_service.cc:1467] Processing CreateTablet for tablet e242d90b5bdc4698a23a8017371e558b (DEFAULT_TABLE table=test-table [id=e761ad179e0f487688e56bc528cffa22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:25:30.146981 10748 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e242d90b5bdc4698a23a8017371e558b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:25:30.146981 10486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e242d90b5bdc4698a23a8017371e558b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:25:30.149048 10828 tablet_bootstrap.cc:492] T e242d90b5bdc4698a23a8017371e558b P 0c5e0a40f69c4c34ac68e46cf2cfb329: Bootstrap starting.
I20260925 07:25:30.149904 10829 tablet_bootstrap.cc:492] T e242d90b5bdc4698a23a8017371e558b P 3f2f6698dd594af1b4130cd54ef1d343: Bootstrap starting.
I20260925 07:25:30.149957 10828 tablet_bootstrap.cc:654] T e242d90b5bdc4698a23a8017371e558b P 0c5e0a40f69c4c34ac68e46cf2cfb329: Neither blocks nor log segments found. Creating new log.
I20260925 07:25:30.150110 10830 tablet_bootstrap.cc:492] T e242d90b5bdc4698a23a8017371e558b P fdae728231294bee91c49f05531682e0: Bootstrap starting.
I20260925 07:25:30.150264 10828 log.cc:824] T e242d90b5bdc4698a23a8017371e558b P 0c5e0a40f69c4c34ac68e46cf2cfb329: Log is configured to *not* fsync() on all Append() calls
I20260925 07:25:30.150702 10830 tablet_bootstrap.cc:654] T e242d90b5bdc4698a23a8017371e558b P fdae728231294bee91c49f05531682e0: Neither blocks nor log segments found. Creating new log.
I20260925 07:25:30.150943 10828 tablet_bootstrap.cc:492] T e242d90b5bdc4698a23a8017371e558b P 0c5e0a40f69c4c34ac68e46cf2cfb329: No bootstrap required, opened a new log
I20260925 07:25:30.150974 10830 log.cc:824] T e242d90b5bdc4698a23a8017371e558b P fdae728231294bee91c49f05531682e0: Log is configured to *not* fsync() on all Append() calls
I20260925 07:25:30.151018 10828 ts_tablet_manager.cc:1397] T e242d90b5bdc4698a23a8017371e558b P 0c5e0a40f69c4c34ac68e46cf2cfb329: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 07:25:30.151698 10829 tablet_bootstrap.cc:654] T e242d90b5bdc4698a23a8017371e558b P 3f2f6698dd594af1b4130cd54ef1d343: Neither blocks nor log segments found. Creating new log.
I20260925 07:25:30.151979 10829 log.cc:824] T e242d90b5bdc4698a23a8017371e558b P 3f2f6698dd594af1b4130cd54ef1d343: Log is configured to *not* fsync() on all Append() calls
I20260925 07:25:30.152498 10830 tablet_bootstrap.cc:492] T e242d90b5bdc4698a23a8017371e558b P fdae728231294bee91c49f05531682e0: No bootstrap required, opened a new log
I20260925 07:25:30.152565 10830 ts_tablet_manager.cc:1397] T e242d90b5bdc4698a23a8017371e558b P fdae728231294bee91c49f05531682e0: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260925 07:25:30.153446 10829 tablet_bootstrap.cc:492] T e242d90b5bdc4698a23a8017371e558b P 3f2f6698dd594af1b4130cd54ef1d343: No bootstrap required, opened a new log
I20260925 07:25:30.153507 10829 ts_tablet_manager.cc:1397] T e242d90b5bdc4698a23a8017371e558b P 3f2f6698dd594af1b4130cd54ef1d343: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260925 07:25:30.153848 10828 raft_consensus.cc:348] T e242d90b5bdc4698a23a8017371e558b P 0c5e0a40f69c4c34ac68e46cf2cfb329 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c5e0a40f69c4c34ac68e46cf2cfb329" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36769 } } peers { permanent_uuid: "fdae728231294bee91c49f05531682e0" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 36889 } } peers { permanent_uuid: "3f2f6698dd594af1b4130cd54ef1d343" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 43661 } }
I20260925 07:25:30.154014 10828 raft_consensus.cc:374] T e242d90b5bdc4698a23a8017371e558b P 0c5e0a40f69c4c34ac68e46cf2cfb329 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:25:30.154048 10828 raft_consensus.cc:729] T e242d90b5bdc4698a23a8017371e558b P 0c5e0a40f69c4c34ac68e46cf2cfb329 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c5e0a40f69c4c34ac68e46cf2cfb329, State: Initialized, Role: FOLLOWER
I20260925 07:25:30.154177 10828 consensus_queue.cc:260] T e242d90b5bdc4698a23a8017371e558b P 0c5e0a40f69c4c34ac68e46cf2cfb329 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c5e0a40f69c4c34ac68e46cf2cfb329" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36769 } } peers { permanent_uuid: "fdae728231294bee91c49f05531682e0" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 36889 } } peers { permanent_uuid: "3f2f6698dd594af1b4130cd54ef1d343" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 43661 } }
I20260925 07:25:30.154466 10683 heartbeater.cc:503] Master 127.5.158.126:44811 was elected leader, sending a full tablet report...
I20260925 07:25:30.154726 10828 ts_tablet_manager.cc:1428] T e242d90b5bdc4698a23a8017371e558b P 0c5e0a40f69c4c34ac68e46cf2cfb329: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260925 07:25:30.154999 10830 raft_consensus.cc:348] T e242d90b5bdc4698a23a8017371e558b P fdae728231294bee91c49f05531682e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c5e0a40f69c4c34ac68e46cf2cfb329" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36769 } } peers { permanent_uuid: "fdae728231294bee91c49f05531682e0" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 36889 } } peers { permanent_uuid: "3f2f6698dd594af1b4130cd54ef1d343" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 43661 } }
I20260925 07:25:30.155154 10830 raft_consensus.cc:374] T e242d90b5bdc4698a23a8017371e558b P fdae728231294bee91c49f05531682e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:25:30.155193 10830 raft_consensus.cc:729] T e242d90b5bdc4698a23a8017371e558b P fdae728231294bee91c49f05531682e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdae728231294bee91c49f05531682e0, State: Initialized, Role: FOLLOWER
I20260925 07:25:30.155292 10830 consensus_queue.cc:260] T e242d90b5bdc4698a23a8017371e558b P fdae728231294bee91c49f05531682e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c5e0a40f69c4c34ac68e46cf2cfb329" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36769 } } peers { permanent_uuid: "fdae728231294bee91c49f05531682e0" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 36889 } } peers { permanent_uuid: "3f2f6698dd594af1b4130cd54ef1d343" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 43661 } }
I20260925 07:25:30.155537 10830 ts_tablet_manager.cc:1428] T e242d90b5bdc4698a23a8017371e558b P fdae728231294bee91c49f05531682e0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 07:25:30.155642 10814 heartbeater.cc:503] Master 127.5.158.126:44811 was elected leader, sending a full tablet report...
I20260925 07:25:30.155665 10829 raft_consensus.cc:348] T e242d90b5bdc4698a23a8017371e558b P 3f2f6698dd594af1b4130cd54ef1d343 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c5e0a40f69c4c34ac68e46cf2cfb329" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36769 } } peers { permanent_uuid: "fdae728231294bee91c49f05531682e0" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 36889 } } peers { permanent_uuid: "3f2f6698dd594af1b4130cd54ef1d343" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 43661 } }
I20260925 07:25:30.155768 10829 raft_consensus.cc:374] T e242d90b5bdc4698a23a8017371e558b P 3f2f6698dd594af1b4130cd54ef1d343 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:25:30.155833 10829 raft_consensus.cc:729] T e242d90b5bdc4698a23a8017371e558b P 3f2f6698dd594af1b4130cd54ef1d343 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f2f6698dd594af1b4130cd54ef1d343, State: Initialized, Role: FOLLOWER
I20260925 07:25:30.155936 10829 consensus_queue.cc:260] T e242d90b5bdc4698a23a8017371e558b P 3f2f6698dd594af1b4130cd54ef1d343 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c5e0a40f69c4c34ac68e46cf2cfb329" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36769 } } peers { permanent_uuid: "fdae728231294bee91c49f05531682e0" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 36889 } } peers { permanent_uuid: "3f2f6698dd594af1b4130cd54ef1d343" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 43661 } }
I20260925 07:25:30.156193 10829 ts_tablet_manager.cc:1428] T e242d90b5bdc4698a23a8017371e558b P 3f2f6698dd594af1b4130cd54ef1d343: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 07:25:30.156234 10552 heartbeater.cc:503] Master 127.5.158.126:44811 was elected leader, sending a full tablet report...
I20260925 07:25:30.160367 10834 raft_consensus.cc:482] T e242d90b5bdc4698a23a8017371e558b P 0c5e0a40f69c4c34ac68e46cf2cfb329 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 07:25:30.160454 10834 raft_consensus.cc:504] T e242d90b5bdc4698a23a8017371e558b P 0c5e0a40f69c4c34ac68e46cf2cfb329 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c5e0a40f69c4c34ac68e46cf2cfb329" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36769 } } peers { permanent_uuid: "fdae728231294bee91c49f05531682e0" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 36889 } } peers { permanent_uuid: "3f2f6698dd594af1b4130cd54ef1d343" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 43661 } }
I20260925 07:25:30.160729 10834 leader_election.cc:288] T e242d90b5bdc4698a23a8017371e558b P 0c5e0a40f69c4c34ac68e46cf2cfb329 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fdae728231294bee91c49f05531682e0 (127.5.158.67:36889), 3f2f6698dd594af1b4130cd54ef1d343 (127.5.158.65:43661)
I20260925 07:25:30.161779 10824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:30.160958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:46605 (local address 127.5.158.65:43661)
0925 07:25:30.160999 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:30.161001 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:30.161008 (+ 7us) server_negotiation.cc:407] Connection header received
0925 07:25:30.161027 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:30.161028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:30.161032 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:30.161099 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:30.161585 (+ 486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:30.161586 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:30.161586 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:25:30.161589 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:30.161601 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:25:30.161608 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:30.161669 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:30.161714 (+ 45us) server_negotiation.cc:299] Negotiation successful
0925 07:25:30.161726 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 07:25:30.161803 10685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:30.160845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:43661 (local address 127.5.158.66:46605)
0925 07:25:30.160881 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:30.160892 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:30.160931 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:30.161123 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:30.161124 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:30.161126 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:30.161499 (+ 373us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 07:25:30.161499 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:30.161505 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 07:25:30.161517 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:30.161675 (+ 158us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:30.161675 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:30.161676 (+ 1us) client_negotiation.cc:769] Sending connection context
0925 07:25:30.161728 (+ 52us) client_negotiation.cc:240] Negotiation successful
0925 07:25:30.161741 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260925 07:25:30.161971 10826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:30.160997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:49511 (local address 127.5.158.67:36889)
0925 07:25:30.161030 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:30.161031 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:30.161067 (+ 36us) server_negotiation.cc:407] Connection header received
0925 07:25:30.161167 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:30.161167 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:30.161171 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:30.161213 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:30.161748 (+ 535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:30.161749 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:30.161749 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:25:30.161753 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:30.161765 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:25:30.161768 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:30.161829 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:30.161903 (+ 74us) server_negotiation.cc:299] Negotiation successful
0925 07:25:30.161915 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 07:25:30.162074 10506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e242d90b5bdc4698a23a8017371e558b" candidate_uuid: "0c5e0a40f69c4c34ac68e46cf2cfb329" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f2f6698dd594af1b4130cd54ef1d343" is_pre_election: true
I20260925 07:25:30.162115 10837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:30.160846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.67:36889 (local address 127.5.158.66:49511)
0925 07:25:30.161019 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:30.161028 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:30.161074 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:30.161216 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:30.161217 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:30.161219 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:30.161663 (+ 444us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 07:25:30.161664 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:30.161669 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 07:25:30.161680 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:30.161831 (+ 151us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:30.161831 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:30.161832 (+ 1us) client_negotiation.cc:769] Sending connection context
0925 07:25:30.162036 (+ 204us) client_negotiation.cc:240] Negotiation successful
0925 07:25:30.162049 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":25,"thread_start_us":73,"threads_started":1}
I20260925 07:25:30.162168 10506 raft_consensus.cc:2454] T e242d90b5bdc4698a23a8017371e558b P 3f2f6698dd594af1b4130cd54ef1d343 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c5e0a40f69c4c34ac68e46cf2cfb329 in term 0.
I20260925 07:25:30.162348 10570 leader_election.cc:302] T e242d90b5bdc4698a23a8017371e558b P 0c5e0a40f69c4c34ac68e46cf2cfb329 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c5e0a40f69c4c34ac68e46cf2cfb329, 3f2f6698dd594af1b4130cd54ef1d343; no voters:
I20260925 07:25:30.162338 10768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e242d90b5bdc4698a23a8017371e558b" candidate_uuid: "0c5e0a40f69c4c34ac68e46cf2cfb329" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdae728231294bee91c49f05531682e0" is_pre_election: true
I20260925 07:25:30.162427 10768 raft_consensus.cc:2454] T e242d90b5bdc4698a23a8017371e558b P fdae728231294bee91c49f05531682e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c5e0a40f69c4c34ac68e46cf2cfb329 in term 0.
I20260925 07:25:30.162449 10834 raft_consensus.cc:2781] T e242d90b5bdc4698a23a8017371e558b P 0c5e0a40f69c4c34ac68e46cf2cfb329 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 07:25:30.162513 10834 raft_consensus.cc:482] T e242d90b5bdc4698a23a8017371e558b P 0c5e0a40f69c4c34ac68e46cf2cfb329 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 07:25:30.162542 10834 raft_consensus.cc:3037] T e242d90b5bdc4698a23a8017371e558b P 0c5e0a40f69c4c34ac68e46cf2cfb329 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:25:30.163095 10834 raft_consensus.cc:504] T e242d90b5bdc4698a23a8017371e558b P 0c5e0a40f69c4c34ac68e46cf2cfb329 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c5e0a40f69c4c34ac68e46cf2cfb329" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36769 } } peers { permanent_uuid: "fdae728231294bee91c49f05531682e0" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 36889 } } peers { permanent_uuid: "3f2f6698dd594af1b4130cd54ef1d343" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 43661 } }
I20260925 07:25:30.163204 10834 leader_election.cc:288] T e242d90b5bdc4698a23a8017371e558b P 0c5e0a40f69c4c34ac68e46cf2cfb329 [CANDIDATE]: Term 1 election: Requested vote from peers fdae728231294bee91c49f05531682e0 (127.5.158.67:36889), 3f2f6698dd594af1b4130cd54ef1d343 (127.5.158.65:43661)
I20260925 07:25:30.163321 10768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e242d90b5bdc4698a23a8017371e558b" candidate_uuid: "0c5e0a40f69c4c34ac68e46cf2cfb329" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdae728231294bee91c49f05531682e0"
I20260925 07:25:30.163342 10506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e242d90b5bdc4698a23a8017371e558b" candidate_uuid: "0c5e0a40f69c4c34ac68e46cf2cfb329" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f2f6698dd594af1b4130cd54ef1d343"
I20260925 07:25:30.163374 10768 raft_consensus.cc:3037] T e242d90b5bdc4698a23a8017371e558b P fdae728231294bee91c49f05531682e0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:25:30.163396 10506 raft_consensus.cc:3037] T e242d90b5bdc4698a23a8017371e558b P 3f2f6698dd594af1b4130cd54ef1d343 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:25:30.163883 10768 raft_consensus.cc:2454] T e242d90b5bdc4698a23a8017371e558b P fdae728231294bee91c49f05531682e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c5e0a40f69c4c34ac68e46cf2cfb329 in term 1.
I20260925 07:25:30.163981 10506 raft_consensus.cc:2454] T e242d90b5bdc4698a23a8017371e558b P 3f2f6698dd594af1b4130cd54ef1d343 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c5e0a40f69c4c34ac68e46cf2cfb329 in term 1.
I20260925 07:25:30.164072 10573 leader_election.cc:302] T e242d90b5bdc4698a23a8017371e558b P 0c5e0a40f69c4c34ac68e46cf2cfb329 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c5e0a40f69c4c34ac68e46cf2cfb329, fdae728231294bee91c49f05531682e0; no voters:
I20260925 07:25:30.164134 10834 raft_consensus.cc:2781] T e242d90b5bdc4698a23a8017371e558b P 0c5e0a40f69c4c34ac68e46cf2cfb329 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:25:30.164240 10834 raft_consensus.cc:686] T e242d90b5bdc4698a23a8017371e558b P 0c5e0a40f69c4c34ac68e46cf2cfb329 [term 1 LEADER]: Becoming Leader. State: Replica: 0c5e0a40f69c4c34ac68e46cf2cfb329, State: Running, Role: LEADER
I20260925 07:25:30.164294 10834 consensus_queue.cc:237] T e242d90b5bdc4698a23a8017371e558b P 0c5e0a40f69c4c34ac68e46cf2cfb329 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c5e0a40f69c4c34ac68e46cf2cfb329" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36769 } } peers { permanent_uuid: "fdae728231294bee91c49f05531682e0" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 36889 } } peers { permanent_uuid: "3f2f6698dd594af1b4130cd54ef1d343" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 43661 } }
I20260925 07:25:30.164968 10363 catalog_manager.cc:5495] T e242d90b5bdc4698a23a8017371e558b P 0c5e0a40f69c4c34ac68e46cf2cfb329 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0c5e0a40f69c4c34ac68e46cf2cfb329 (127.5.158.66). New cstate: current_term: 1 leader_uuid: "0c5e0a40f69c4c34ac68e46cf2cfb329" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c5e0a40f69c4c34ac68e46cf2cfb329" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36769 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fdae728231294bee91c49f05531682e0" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 36889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f2f6698dd594af1b4130cd54ef1d343" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 43661 } health_report { overall_health: UNKNOWN } } }
W20260925 07:25:30.169479 10553 tablet.cc:2369] T e242d90b5bdc4698a23a8017371e558b P 3f2f6698dd594af1b4130cd54ef1d343: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 07:25:30.173853 10827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:30.173241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51302 (local address 127.5.158.66:36769)
0925 07:25:30.173269 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:30.173271 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:30.173311 (+ 40us) server_negotiation.cc:407] Connection header received
0925 07:25:30.173348 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:30.173349 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:30.173356 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:30.173395 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:30.173729 (+ 334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:30.173730 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:30.173730 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:25:30.173733 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:30.173742 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:25:30.173745 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:30.173786 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:30.173812 (+ 26us) server_negotiation.cc:299] Negotiation successful
0925 07:25:30.173820 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
I20260925 07:25:30.175097 10768 raft_consensus.cc:1261] T e242d90b5bdc4698a23a8017371e558b P fdae728231294bee91c49f05531682e0 [term 1 FOLLOWER]: Refusing update from remote peer 0c5e0a40f69c4c34ac68e46cf2cfb329: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:25:30.175102 10506 raft_consensus.cc:1261] T e242d90b5bdc4698a23a8017371e558b P 3f2f6698dd594af1b4130cd54ef1d343 [term 1 FOLLOWER]: Refusing update from remote peer 0c5e0a40f69c4c34ac68e46cf2cfb329: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:25:30.175405 10838 consensus_queue.cc:1035] T e242d90b5bdc4698a23a8017371e558b P 0c5e0a40f69c4c34ac68e46cf2cfb329 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fdae728231294bee91c49f05531682e0" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 36889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:25:30.175510 10834 consensus_queue.cc:1035] T e242d90b5bdc4698a23a8017371e558b P 0c5e0a40f69c4c34ac68e46cf2cfb329 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f2f6698dd594af1b4130cd54ef1d343" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 43661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:25:30.177297 10841 mvcc.cc:205] Tried to move back new op lower bound from 7333155349195591680 to 7333155349153832960. Current Snapshot: MvccSnapshot[applied={T|T < 7333155349195591680}]
I20260925 07:25:30.177879 10843 mvcc.cc:205] Tried to move back new op lower bound from 7333155349195591680 to 7333155349153832960. Current Snapshot: MvccSnapshot[applied={T|T < 7333155349195591680}]
I20260925 07:25:30.178395 10842 mvcc.cc:205] Tried to move back new op lower bound from 7333155349195591680 to 7333155349153832960. Current Snapshot: MvccSnapshot[applied={T|T < 7333155349195591680 or (T in {7333155349195591680})}]
I20260925 07:25:30.179433 10364 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37774:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 07:25:30.179533 10364 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37774:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 07:25:30.180063 10364 catalog_manager.cc:5782] T 00000000000000000000000000000000 P acb46a295e51498d8f0a3a9bbe742184: Sending DeleteTablet for 3 replicas of tablet e242d90b5bdc4698a23a8017371e558b
I20260925 07:25:30.180301 10617 tablet_service.cc:1514] Processing DeleteTablet for tablet e242d90b5bdc4698a23a8017371e558b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:25:30 UTC) from {username='slave'} at 127.0.0.1:51286
I20260925 07:25:30.180356 10486 tablet_service.cc:1514] Processing DeleteTablet for tablet e242d90b5bdc4698a23a8017371e558b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:25:30 UTC) from {username='slave'} at 127.0.0.1:46132
I20260925 07:25:30.180532 10852 tablet_replica.cc:330] T e242d90b5bdc4698a23a8017371e558b P 3f2f6698dd594af1b4130cd54ef1d343: stopping tablet replica
I20260925 07:25:30.180539 10748 tablet_service.cc:1514] Processing DeleteTablet for tablet e242d90b5bdc4698a23a8017371e558b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:25:30 UTC) from {username='slave'} at 127.0.0.1:48498
I20260925 07:25:30.180563 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 10424
I20260925 07:25:30.180605 10852 raft_consensus.cc:2229] T e242d90b5bdc4698a23a8017371e558b P 3f2f6698dd594af1b4130cd54ef1d343 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 07:25:30.180677 10852 raft_consensus.cc:2258] T e242d90b5bdc4698a23a8017371e558b P 3f2f6698dd594af1b4130cd54ef1d343 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:25:30.180804 10853 tablet_replica.cc:330] T e242d90b5bdc4698a23a8017371e558b P fdae728231294bee91c49f05531682e0: stopping tablet replica
I20260925 07:25:30.180907 10853 raft_consensus.cc:2229] T e242d90b5bdc4698a23a8017371e558b P fdae728231294bee91c49f05531682e0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 07:25:30.181074 10853 raft_consensus.cc:2258] T e242d90b5bdc4698a23a8017371e558b P fdae728231294bee91c49f05531682e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:25:30.181495 10853 ts_tablet_manager.cc:1905] T e242d90b5bdc4698a23a8017371e558b P fdae728231294bee91c49f05531682e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:25:30.183030 10853 ts_tablet_manager.cc:1918] T e242d90b5bdc4698a23a8017371e558b P fdae728231294bee91c49f05531682e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 07:25:30.183094 10853 log.cc:1196] T e242d90b5bdc4698a23a8017371e558b P fdae728231294bee91c49f05531682e0: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/ts-2/wal/wals/e242d90b5bdc4698a23a8017371e558b
I20260925 07:25:30.183338 10853 ts_tablet_manager.cc:1939] T e242d90b5bdc4698a23a8017371e558b P fdae728231294bee91c49f05531682e0: Deleting consensus metadata
I20260925 07:25:30.183687 10351 catalog_manager.cc:4841] TS fdae728231294bee91c49f05531682e0 (127.5.158.67:36889): tablet e242d90b5bdc4698a23a8017371e558b (table test-table [id=e761ad179e0f487688e56bc528cffa22]) successfully deleted
I20260925 07:25:30.185225 10851 tablet_replica.cc:330] T e242d90b5bdc4698a23a8017371e558b P 0c5e0a40f69c4c34ac68e46cf2cfb329: stopping tablet replica
I20260925 07:25:30.185308 10851 raft_consensus.cc:2229] T e242d90b5bdc4698a23a8017371e558b P 0c5e0a40f69c4c34ac68e46cf2cfb329 [term 1 LEADER]: Raft consensus shutting down.
I20260925 07:25:30.185379 10851 raft_consensus.cc:2258] T e242d90b5bdc4698a23a8017371e558b P 0c5e0a40f69c4c34ac68e46cf2cfb329 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:25:30.185729 10851 ts_tablet_manager.cc:1905] T e242d90b5bdc4698a23a8017371e558b P 0c5e0a40f69c4c34ac68e46cf2cfb329: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 07:25:30.186688 10348 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.158.65:43661 (error 108)
I20260925 07:25:30.187086 10822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:30.186986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:43661 (local address 127.0.0.1:46134)
0925 07:25:30.187024 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:30.187056 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.65:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260925 07:25:30.187093 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 10555
I20260925 07:25:30.187099 10851 ts_tablet_manager.cc:1918] T e242d90b5bdc4698a23a8017371e558b P 0c5e0a40f69c4c34ac68e46cf2cfb329: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 07:25:30.187150 10851 log.cc:1196] T e242d90b5bdc4698a23a8017371e558b P 0c5e0a40f69c4c34ac68e46cf2cfb329: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790321110323796-5753-0/minicluster-data/ts-1/wal/wals/e242d90b5bdc4698a23a8017371e558b
I20260925 07:25:30.187381 10851 ts_tablet_manager.cc:1939] T e242d90b5bdc4698a23a8017371e558b P 0c5e0a40f69c4c34ac68e46cf2cfb329: Deleting consensus metadata
W20260925 07:25:30.187706 10348 catalog_manager.cc:4584] TS 3f2f6698dd594af1b4130cd54ef1d343 (127.5.158.65:43661): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e242d90b5bdc4698a23a8017371e558b: Network error: Client connection negotiation failed: client connection to 127.5.158.65:43661: connect: Connection refused (error 111)
I20260925 07:25:30.192127 10822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:30.192032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.66:36769 (local address 127.0.0.1:51308)
0925 07:25:30.192079 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:30.192108 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.66:36769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260925 07:25:30.192128 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 10686
I20260925 07:25:30.196066 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 10333
2026-09-25T07:25:30Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (522 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260925 07:25:30.215135 5753 mini_oidc.cc:152] Starting JWKS server
I20260925 07:25:30.216028 5753 webserver.cc:460] Webserver started at https://127.0.0.1:46769/ using document root <none> and password file <none>
I20260925 07:25:30.216133 5753 mini_oidc.cc:189] Starting OIDC Discovery server
I20260925 07:25:30.216310 5753 webserver.cc:460] Webserver started at http://127.0.0.1:43231/ using document root <none> and password file <none>
2026-09-25T07:25:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T07:25:30Z Disabled control of system clock
I20260925 07:25:30.222702 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:34737
--webserver_interface=127.5.158.126
--webserver_port=0
--builtin_ntp_servers=127.5.158.84:36001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.158.126:34737
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46769/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790321110323796-5753-0/testchainca.cert with env {}
W20260925 07:25:30.312273 10860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:25:30.312433 10860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:25:30.312453 10860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:25:30.313524 10860 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:25:30.313565 10860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:25:30.313578 10860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:25:30.313591 10860 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:25:30.313603 10860 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260925 07:25:30.313621 10860 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260925 07:25:30.313638 10860 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46769/jwks.json
I20260925 07:25:30.314913 10860 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:36001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790321110323796-5753-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.158.126:34737
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:34737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46769/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.158.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790321110323796-5753-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10860
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:25:30.315075 10860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:25:30.315222 10860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 07:25:30.319676 10860 server_base.cc:1034] running on GCE node
W20260925 07:25:30.319720 10865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:30.319687 10866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:30.319828 10868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:30.320071 10860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:25:30.320340 10860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:25:30.321489 10860 hybrid_clock.cc:648] HybridClock initialized: now 1790321130321477 us; error 26 us; skew 500 ppm
I20260925 07:25:30.322577 10860 webserver.cc:460] Webserver started at http://127.5.158.126:33017/ using document root <none> and password file <none>
I20260925 07:25:30.322777 10860 fs_manager.cc:359] Metadata directory not provided
I20260925 07:25:30.322834 10860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:25:30.322947 10860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:25:30.323793 10860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790321110323796-5753-0/minicluster-data/master-0/data/instance:
uuid: "eb502d525513467b8c72d81cfe62b047"
format_stamp: "Formatted at 2026-09-25 07:25:30 on dist-test-slave-1drq"
server_key: "a0685edb44900c98f129731a202d5bee"
server_key_iv: "edb533a436b739df55c4e93057dc5085"
server_key_version: "encryptionkey@0"
I20260925 07:25:30.324107 10860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790321110323796-5753-0/minicluster-data/master-0/wal/instance:
uuid: "eb502d525513467b8c72d81cfe62b047"
format_stamp: "Formatted at 2026-09-25 07:25:30 on dist-test-slave-1drq"
server_key: "a0685edb44900c98f129731a202d5bee"
server_key_iv: "edb533a436b739df55c4e93057dc5085"
server_key_version: "encryptionkey@0"
I20260925 07:25:30.325397 10860 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 07:25:30.326040 10874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:30.326184 10860 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:25:30.326265 10860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "eb502d525513467b8c72d81cfe62b047"
format_stamp: "Formatted at 2026-09-25 07:25:30 on dist-test-slave-1drq"
server_key: "a0685edb44900c98f129731a202d5bee"
server_key_iv: "edb533a436b739df55c4e93057dc5085"
server_key_version: "encryptionkey@0"
I20260925 07:25:30.326349 10860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:25:30.341971 10860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:25:30.342198 10860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:25:30.342327 10860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:25:30.346730 10860 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.126:34737
I20260925 07:25:30.346760 10926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.126:34737 every 8 connection(s)
I20260925 07:25:30.347035 10860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:25:30.347890 10927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:25:30.349879 10927 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb502d525513467b8c72d81cfe62b047: Bootstrap starting.
I20260925 07:25:30.350382 10927 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb502d525513467b8c72d81cfe62b047: Neither blocks nor log segments found. Creating new log.
I20260925 07:25:30.350637 10927 log.cc:824] T 00000000000000000000000000000000 P eb502d525513467b8c72d81cfe62b047: Log is configured to *not* fsync() on all Append() calls
I20260925 07:25:30.351191 10927 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb502d525513467b8c72d81cfe62b047: No bootstrap required, opened a new log
I20260925 07:25:30.352568 10927 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb502d525513467b8c72d81cfe62b047 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb502d525513467b8c72d81cfe62b047" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 34737 } }
I20260925 07:25:30.352680 10927 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb502d525513467b8c72d81cfe62b047 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:25:30.352731 10927 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb502d525513467b8c72d81cfe62b047 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb502d525513467b8c72d81cfe62b047, State: Initialized, Role: FOLLOWER
I20260925 07:25:30.352885 10927 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb502d525513467b8c72d81cfe62b047 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb502d525513467b8c72d81cfe62b047" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 34737 } }
I20260925 07:25:30.352957 10927 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb502d525513467b8c72d81cfe62b047 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 07:25:30.353003 10927 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb502d525513467b8c72d81cfe62b047 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 07:25:30.353049 10927 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb502d525513467b8c72d81cfe62b047 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:25:30.353497 10927 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb502d525513467b8c72d81cfe62b047 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb502d525513467b8c72d81cfe62b047" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 34737 } }
I20260925 07:25:30.353602 10927 leader_election.cc:302] T 00000000000000000000000000000000 P eb502d525513467b8c72d81cfe62b047 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb502d525513467b8c72d81cfe62b047; no voters:
I20260925 07:25:30.353761 10927 leader_election.cc:288] T 00000000000000000000000000000000 P eb502d525513467b8c72d81cfe62b047 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 07:25:30.353798 10930 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb502d525513467b8c72d81cfe62b047 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:25:30.353955 10930 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb502d525513467b8c72d81cfe62b047 [term 1 LEADER]: Becoming Leader. State: Replica: eb502d525513467b8c72d81cfe62b047, State: Running, Role: LEADER
I20260925 07:25:30.353986 10927 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb502d525513467b8c72d81cfe62b047 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:25:30.354040 10930 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb502d525513467b8c72d81cfe62b047 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb502d525513467b8c72d81cfe62b047" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 34737 } }
I20260925 07:25:30.354449 10932 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb502d525513467b8c72d81cfe62b047 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb502d525513467b8c72d81cfe62b047. Latest consensus state: current_term: 1 leader_uuid: "eb502d525513467b8c72d81cfe62b047" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb502d525513467b8c72d81cfe62b047" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 34737 } } }
I20260925 07:25:30.354549 10932 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb502d525513467b8c72d81cfe62b047 [sys.catalog]: This master's current role is: LEADER
I20260925 07:25:30.354719 10931 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb502d525513467b8c72d81cfe62b047 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eb502d525513467b8c72d81cfe62b047" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb502d525513467b8c72d81cfe62b047" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 34737 } } }
I20260925 07:25:30.354804 10931 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb502d525513467b8c72d81cfe62b047 [sys.catalog]: This master's current role is: LEADER
I20260925 07:25:30.354933 10939 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 07:25:30.355366 10939 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 07:25:30.356329 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 10860
I20260925 07:25:30.356413 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790321110323796-5753-0/minicluster-data/master-0/wal/instance
I20260925 07:25:30.356578 5753 external_mini_cluster.cc:1428] Setting key 8a4274f16eba26b2db0359300a0771c4
I20260925 07:25:30.357213 10939 catalog_manager.cc:1347] Generated new cluster ID: 3b56379915524d8a8f34a7201c085cae
I20260925 07:25:30.357257 10939 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 07:25:30.368721 10950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:30.356928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43772 (local address 127.5.158.126:34737)
0925 07:25:30.357106 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:30.357115 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:30.357178 (+ 63us) server_negotiation.cc:407] Connection header received
0925 07:25:30.364181 (+ 7003us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:30.364189 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:30.364248 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:30.364396 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:30.364552 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:30.365847 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:30.366728 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:30.367306 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:30.367457 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:30.368085 (+ 628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:30.368089 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:30.368089 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:25:30.368286 (+ 197us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:30.368328 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:25:30.368341 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:30.368403 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:30.368476 (+ 73us) server_negotiation.cc:299] Negotiation successful
0925 07:25:30.368549 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":91,"thread_start_us":68,"threads_started":1}
I20260925 07:25:30.369534 5753 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 07:25:30.372133 10939 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 07:25:30.372538 10939 catalog_manager.cc:1504] Loading token signing keys...
I20260925 07:25:30.377921 10939 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb502d525513467b8c72d81cfe62b047: Generated new TSK 0
I20260925 07:25:30.378069 10939 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260925 07:25:30.397717 10957 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260925 07:25:30.397795 10957 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260925 07:25:30.397996 10950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:30.384899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43776 (local address 127.5.158.126:34737)
0925 07:25:30.384958 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:30.384960 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:30.385072 (+ 112us) server_negotiation.cc:407] Connection header received
0925 07:25:30.385180 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:30.385180 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:30.385189 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:30.385239 (+ 50us) server_negotiation.cc:226] Negotiated authn=JWT
0925 07:25:30.385354 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:30.385726 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:30.386662 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:30.386802 (+ 140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:30.386841 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:30.387008 (+ 167us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0925 07:25:30.397819 (+ 10811us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46769/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0925 07:25:30.397957 (+ 138us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43776: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46769/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":28}
W20260925 07:25:30.398054 10950 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43776: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46769/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260925 07:25:30.398406 10956 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:25:30.384850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:34737 (local address 127.0.0.1:43776)
0925 07:25:30.385001 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:30.385018 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:30.385098 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:30.385251 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:30.385252 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:30.385253 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0925 07:25:30.385304 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:30.385305 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:30.385788 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:30.385789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:30.386598 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:30.386600 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:30.386859 (+ 259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:30.386860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:30.386912 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:30.386931 (+ 19us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0925 07:25:30.397940 (+ 11009us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46769/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0925 07:25:30.398053 (+ 113us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.158.126:34737: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46769/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":80,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.5.158.126:34737: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46769/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260925 07:25:30.398816 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 10860
2026-09-25T07:25:30Z chronyd exiting
I20260925 07:25:30.433920 5753 test_util.cc:175] -----------------------------------------------
I20260925 07:25:30.433985 5753 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790321110323796-5753-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (219 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260925 07:25:30.434659 5753 mini_oidc.cc:152] Starting JWKS server
I20260925 07:25:30.435205 5753 webserver.cc:460] Webserver started at https://127.0.0.1:42841/ using document root <none> and password file <none>
I20260925 07:25:30.435276 5753 mini_oidc.cc:189] Starting OIDC Discovery server
I20260925 07:25:30.435431 5753 webserver.cc:460] Webserver started at http://127.0.0.1:35261/ using document root <none> and password file <none>
2026-09-25T07:25:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T07:25:30Z Disabled control of system clock
I20260925 07:25:30.441955 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:45609
--webserver_interface=127.5.158.126
--webserver_port=0
--builtin_ntp_servers=127.5.158.84:33249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.158.126:45609
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42841/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790321110323796-5753-0/testchainca.cert with env {}
W20260925 07:25:30.512236 10963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:25:30.512394 10963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:25:30.512413 10963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:25:30.513370 10963 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:25:30.513403 10963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:25:30.513423 10963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:25:30.513434 10963 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:25:30.513444 10963 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260925 07:25:30.513454 10963 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260925 07:25:30.513473 10963 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42841/jwks.json
I20260925 07:25:30.514657 10963 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:33249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790321110323796-5753-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.158.126:45609
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:45609
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42841/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.158.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790321110323796-5753-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10963
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:25:30.514824 10963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:25:30.515007 10963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:25:30.519034 10968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:30.519150 10971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:30.519292 10969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:30.519264 10963 server_base.cc:1034] running on GCE node
I20260925 07:25:30.519483 10963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:25:30.519695 10963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:25:30.520807 10963 hybrid_clock.cc:648] HybridClock initialized: now 1790321130520798 us; error 25 us; skew 500 ppm
I20260925 07:25:30.521813 10963 webserver.cc:460] Webserver started at http://127.5.158.126:35493/ using document root <none> and password file <none>
I20260925 07:25:30.521982 10963 fs_manager.cc:359] Metadata directory not provided
I20260925 07:25:30.522019 10963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:25:30.522090 10963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:25:30.522897 10963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790321110323796-5753-0/minicluster-data/master-0/data/instance:
uuid: "42f30dde8a3e457f83e4f289b7edaa21"
format_stamp: "Formatted at 2026-09-25 07:25:30 on dist-test-slave-1drq"
server_key: "05813dc768efb8ce4e57ce5d523431ac"
server_key_iv: "17918415d6cdf31ae4d5e29669a77059"
server_key_version: "encryptionkey@0"
I20260925 07:25:30.523178 10963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790321110323796-5753-0/minicluster-data/master-0/wal/instance:
uuid: "42f30dde8a3e457f83e4f289b7edaa21"
format_stamp: "Formatted at 2026-09-25 07:25:30 on dist-test-slave-1drq"
server_key: "05813dc768efb8ce4e57ce5d523431ac"
server_key_iv: "17918415d6cdf31ae4d5e29669a77059"
server_key_version: "encryptionkey@0"
I20260925 07:25:30.524232 10963 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 07:25:30.524783 10977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:30.524931 10963 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 07:25:30.525014 10963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "42f30dde8a3e457f83e4f289b7edaa21"
format_stamp: "Formatted at 2026-09-25 07:25:30 on dist-test-slave-1drq"
server_key: "05813dc768efb8ce4e57ce5d523431ac"
server_key_iv: "17918415d6cdf31ae4d5e29669a77059"
server_key_version: "encryptionkey@0"
I20260925 07:25:30.525081 10963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:25:30.554903 10963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:25:30.555145 10963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:25:30.555264 10963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:25:30.558650 10963 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.126:45609
I20260925 07:25:30.558691 11029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.126:45609 every 8 connection(s)
I20260925 07:25:30.559001 10963 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:25:30.559872 11030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:25:30.561817 11030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42f30dde8a3e457f83e4f289b7edaa21: Bootstrap starting.
I20260925 07:25:30.562307 11030 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 42f30dde8a3e457f83e4f289b7edaa21: Neither blocks nor log segments found. Creating new log.
I20260925 07:25:30.562556 11030 log.cc:824] T 00000000000000000000000000000000 P 42f30dde8a3e457f83e4f289b7edaa21: Log is configured to *not* fsync() on all Append() calls
I20260925 07:25:30.563119 11030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42f30dde8a3e457f83e4f289b7edaa21: No bootstrap required, opened a new log
I20260925 07:25:30.564416 11030 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42f30dde8a3e457f83e4f289b7edaa21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42f30dde8a3e457f83e4f289b7edaa21" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 45609 } }
I20260925 07:25:30.564527 11030 raft_consensus.cc:374] T 00000000000000000000000000000000 P 42f30dde8a3e457f83e4f289b7edaa21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:25:30.564572 11030 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42f30dde8a3e457f83e4f289b7edaa21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42f30dde8a3e457f83e4f289b7edaa21, State: Initialized, Role: FOLLOWER
I20260925 07:25:30.564688 11030 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42f30dde8a3e457f83e4f289b7edaa21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42f30dde8a3e457f83e4f289b7edaa21" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 45609 } }
I20260925 07:25:30.564740 11030 raft_consensus.cc:388] T 00000000000000000000000000000000 P 42f30dde8a3e457f83e4f289b7edaa21 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 07:25:30.564774 11030 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42f30dde8a3e457f83e4f289b7edaa21 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 07:25:30.564815 11030 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42f30dde8a3e457f83e4f289b7edaa21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:25:30.565279 11030 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42f30dde8a3e457f83e4f289b7edaa21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42f30dde8a3e457f83e4f289b7edaa21" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 45609 } }
I20260925 07:25:30.565371 11030 leader_election.cc:302] T 00000000000000000000000000000000 P 42f30dde8a3e457f83e4f289b7edaa21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42f30dde8a3e457f83e4f289b7edaa21; no voters:
I20260925 07:25:30.565534 11030 leader_election.cc:288] T 00000000000000000000000000000000 P 42f30dde8a3e457f83e4f289b7edaa21 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 07:25:30.565558 11033 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42f30dde8a3e457f83e4f289b7edaa21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:25:30.565583 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 10963
I20260925 07:25:30.565672 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790321110323796-5753-0/minicluster-data/master-0/wal/instance
I20260925 07:25:30.565698 11033 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42f30dde8a3e457f83e4f289b7edaa21 [term 1 LEADER]: Becoming Leader. State: Replica: 42f30dde8a3e457f83e4f289b7edaa21, State: Running, Role: LEADER
I20260925 07:25:30.565763 11030 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42f30dde8a3e457f83e4f289b7edaa21 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:25:30.565778 11033 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42f30dde8a3e457f83e4f289b7edaa21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42f30dde8a3e457f83e4f289b7edaa21" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 45609 } }
I20260925 07:25:30.565873 5753 external_mini_cluster.cc:1428] Setting key 2fab17ed42c592e4647de477781e1b86
I20260925 07:25:30.566062 11034 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42f30dde8a3e457f83e4f289b7edaa21 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "42f30dde8a3e457f83e4f289b7edaa21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42f30dde8a3e457f83e4f289b7edaa21" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 45609 } } }
I20260925 07:25:30.566141 11034 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42f30dde8a3e457f83e4f289b7edaa21 [sys.catalog]: This master's current role is: LEADER
I20260925 07:25:30.566392 11035 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42f30dde8a3e457f83e4f289b7edaa21 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42f30dde8a3e457f83e4f289b7edaa21. Latest consensus state: current_term: 1 leader_uuid: "42f30dde8a3e457f83e4f289b7edaa21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42f30dde8a3e457f83e4f289b7edaa21" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 45609 } } }
I20260925 07:25:30.566502 11035 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42f30dde8a3e457f83e4f289b7edaa21 [sys.catalog]: This master's current role is: LEADER
I20260925 07:25:30.566433 11039 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 07:25:30.567322 11039 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 07:25:30.568943 11039 catalog_manager.cc:1347] Generated new cluster ID: 31af28aa716947249e94b9e72757f744
I20260925 07:25:30.568990 11039 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 07:25:30.571164 11042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:30.566258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44396 (local address 127.5.158.126:45609)
0925 07:25:30.566536 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:30.566544 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:30.566575 (+ 31us) server_negotiation.cc:407] Connection header received
0925 07:25:30.566777 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:30.566784 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:30.566841 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:30.566960 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:30.567272 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:30.568678 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:30.569445 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:30.569830 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:30.569945 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:30.570524 (+ 579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:30.570528 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:30.570528 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:25:30.570679 (+ 151us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:30.570706 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:25:30.570715 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:30.570782 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:30.570860 (+ 78us) server_negotiation.cc:299] Negotiation successful
0925 07:25:30.570999 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":75,"threads_started":1}
I20260925 07:25:30.571832 5753 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 07:25:30.581040 11039 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 07:25:30.581609 11039 catalog_manager.cc:1504] Loading token signing keys...
I20260925 07:25:30.588820 11039 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 42f30dde8a3e457f83e4f289b7edaa21: Generated new TSK 0
I20260925 07:25:30.588963 11039 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260925 07:25:30.609771 11060 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260925 07:25:30.609843 11060 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260925 07:25:30.610087 11042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:30.597194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44398 (local address 127.5.158.126:45609)
0925 07:25:30.597239 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:30.597241 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:30.597366 (+ 125us) server_negotiation.cc:407] Connection header received
0925 07:25:30.597443 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:30.597444 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:30.597455 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:30.597526 (+ 71us) server_negotiation.cc:226] Negotiated authn=JWT
0925 07:25:30.597611 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:30.598378 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:30.599065 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:30.599351 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:30.599424 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:30.599613 (+ 189us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0925 07:25:30.609921 (+ 10308us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42841/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0925 07:25:30.610024 (+ 103us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44398: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42841/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":28}
W20260925 07:25:30.610131 11042 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44398: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42841/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260925 07:25:30.610158 11059 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:25:30.597138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:45609 (local address 127.0.0.1:44398)
0925 07:25:30.597289 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:30.597298 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:30.597368 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:30.597525 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:30.597525 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:30.597527 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0925 07:25:30.597570 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:30.597572 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:30.598447 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:30.598448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:30.598994 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:30.598995 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:30.599447 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:30.599448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:30.599519 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:30.599523 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0925 07:25:30.610012 (+ 10489us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42841/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0925 07:25:30.610077 (+ 65us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.158.126:45609: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42841/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":73,"threads_started":1}
I20260925 07:25:30.610379 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 10963
2026-09-25T07:25:30Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (212 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260925 07:25:30.646973 5753 mini_oidc.cc:152] Starting JWKS server
I20260925 07:25:30.647586 5753 webserver.cc:460] Webserver started at https://127.0.0.1:34105/ using document root <none> and password file <none>
I20260925 07:25:30.647624 5753 mini_oidc.cc:189] Starting OIDC Discovery server
I20260925 07:25:30.647786 5753 webserver.cc:460] Webserver started at http://127.0.0.1:41227/ using document root <none> and password file <none>
2026-09-25T07:25:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T07:25:30Z Disabled control of system clock
I20260925 07:25:30.654033 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:46115
--webserver_interface=127.5.158.126
--webserver_port=0
--builtin_ntp_servers=127.5.158.84:41381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.158.126:46115
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34105/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260925 07:25:30.725004 11066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:25:30.725215 11066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:25:30.725250 11066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:25:30.726270 11066 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:25:30.726323 11066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:25:30.726341 11066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:25:30.726361 11066 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:25:30.726387 11066 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260925 07:25:30.726406 11066 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260925 07:25:30.726432 11066 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34105/jwks.json
I20260925 07:25:30.727734 11066 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:41381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790321110323796-5753-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.158.126:46115
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:46115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34105/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11066
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:25:30.727985 11066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:25:30.728238 11066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:25:30.732400 11072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:30.732436 11066 server_base.cc:1034] running on GCE node
W20260925 07:25:30.732424 11071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:30.732442 11074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:30.732751 11066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:25:30.732995 11066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:25:30.734136 11066 hybrid_clock.cc:648] HybridClock initialized: now 1790321130734105 us; error 56 us; skew 500 ppm
I20260925 07:25:30.735209 11066 webserver.cc:460] Webserver started at http://127.5.158.126:35409/ using document root <none> and password file <none>
I20260925 07:25:30.735404 11066 fs_manager.cc:359] Metadata directory not provided
I20260925 07:25:30.735459 11066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:25:30.735613 11066 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:25:30.736567 11066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790321110323796-5753-0/minicluster-data/master-0/data/instance:
uuid: "ad8ac9bf1f5a4c21bcecb65f630798d7"
format_stamp: "Formatted at 2026-09-25 07:25:30 on dist-test-slave-1drq"
server_key: "64377494157f519f93ceb72d2e1a2cba"
server_key_iv: "d1fb8eed1dfafae1e5dd18ea3a6ba1aa"
server_key_version: "encryptionkey@0"
I20260925 07:25:30.736910 11066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790321110323796-5753-0/minicluster-data/master-0/wal/instance:
uuid: "ad8ac9bf1f5a4c21bcecb65f630798d7"
format_stamp: "Formatted at 2026-09-25 07:25:30 on dist-test-slave-1drq"
server_key: "64377494157f519f93ceb72d2e1a2cba"
server_key_iv: "d1fb8eed1dfafae1e5dd18ea3a6ba1aa"
server_key_version: "encryptionkey@0"
I20260925 07:25:30.738163 11066 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 07:25:30.738811 11080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:30.738945 11066 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 07:25:30.739037 11066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "ad8ac9bf1f5a4c21bcecb65f630798d7"
format_stamp: "Formatted at 2026-09-25 07:25:30 on dist-test-slave-1drq"
server_key: "64377494157f519f93ceb72d2e1a2cba"
server_key_iv: "d1fb8eed1dfafae1e5dd18ea3a6ba1aa"
server_key_version: "encryptionkey@0"
I20260925 07:25:30.739111 11066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:25:30.762351 11066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:25:30.762562 11066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:25:30.762681 11066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:25:30.766141 11066 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.126:46115
I20260925 07:25:30.766168 11132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.126:46115 every 8 connection(s)
I20260925 07:25:30.766448 11066 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:25:30.766985 11133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:25:30.767786 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 11066
I20260925 07:25:30.767886 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790321110323796-5753-0/minicluster-data/master-0/wal/instance
I20260925 07:25:30.768069 5753 external_mini_cluster.cc:1428] Setting key 4e1d5ebe3f557bb5b9e49d0704300690
I20260925 07:25:30.769990 11133 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad8ac9bf1f5a4c21bcecb65f630798d7: Bootstrap starting.
I20260925 07:25:30.770576 11133 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad8ac9bf1f5a4c21bcecb65f630798d7: Neither blocks nor log segments found. Creating new log.
I20260925 07:25:30.770900 11133 log.cc:824] T 00000000000000000000000000000000 P ad8ac9bf1f5a4c21bcecb65f630798d7: Log is configured to *not* fsync() on all Append() calls
I20260925 07:25:30.771503 11133 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad8ac9bf1f5a4c21bcecb65f630798d7: No bootstrap required, opened a new log
I20260925 07:25:30.773028 11136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:30.768488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42714 (local address 127.5.158.126:46115)
0925 07:25:30.768680 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:30.768686 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:30.768706 (+ 20us) server_negotiation.cc:407] Connection header received
0925 07:25:30.769050 (+ 344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:30.769055 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:30.769171 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:30.769267 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:30.769502 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:30.770328 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:30.771363 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:30.771742 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:30.771820 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:30.772351 (+ 531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:30.772353 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:30.772354 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:25:30.772490 (+ 136us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:30.772515 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:25:30.772524 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:30.772567 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:30.772655 (+ 88us) server_negotiation.cc:299] Negotiation successful
0925 07:25:30.772724 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":93,"threads_started":1}
I20260925 07:25:30.773166 11133 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad8ac9bf1f5a4c21bcecb65f630798d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad8ac9bf1f5a4c21bcecb65f630798d7" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 46115 } }
I20260925 07:25:30.773309 11133 raft_consensus.cc:374] T 00000000000000000000000000000000 P ad8ac9bf1f5a4c21bcecb65f630798d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:25:30.773340 11133 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad8ac9bf1f5a4c21bcecb65f630798d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad8ac9bf1f5a4c21bcecb65f630798d7, State: Initialized, Role: FOLLOWER
I20260925 07:25:30.773443 11133 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad8ac9bf1f5a4c21bcecb65f630798d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad8ac9bf1f5a4c21bcecb65f630798d7" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 46115 } }
I20260925 07:25:30.773517 11133 raft_consensus.cc:388] T 00000000000000000000000000000000 P ad8ac9bf1f5a4c21bcecb65f630798d7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 07:25:30.773569 11133 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad8ac9bf1f5a4c21bcecb65f630798d7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 07:25:30.773604 11133 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad8ac9bf1f5a4c21bcecb65f630798d7 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:25:30.773895 5753 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 07:25:30.774225 11133 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad8ac9bf1f5a4c21bcecb65f630798d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad8ac9bf1f5a4c21bcecb65f630798d7" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 46115 } }
I20260925 07:25:30.774325 11133 leader_election.cc:302] T 00000000000000000000000000000000 P ad8ac9bf1f5a4c21bcecb65f630798d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad8ac9bf1f5a4c21bcecb65f630798d7; no voters:
I20260925 07:25:30.774480 11133 leader_election.cc:288] T 00000000000000000000000000000000 P ad8ac9bf1f5a4c21bcecb65f630798d7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 07:25:30.774533 11138 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad8ac9bf1f5a4c21bcecb65f630798d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:25:30.774680 11138 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad8ac9bf1f5a4c21bcecb65f630798d7 [term 1 LEADER]: Becoming Leader. State: Replica: ad8ac9bf1f5a4c21bcecb65f630798d7, State: Running, Role: LEADER
I20260925 07:25:30.774681 11133 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad8ac9bf1f5a4c21bcecb65f630798d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:25:30.774758 11138 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad8ac9bf1f5a4c21bcecb65f630798d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad8ac9bf1f5a4c21bcecb65f630798d7" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 46115 } }
I20260925 07:25:30.775068 11139 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad8ac9bf1f5a4c21bcecb65f630798d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ad8ac9bf1f5a4c21bcecb65f630798d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad8ac9bf1f5a4c21bcecb65f630798d7" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 46115 } } }
I20260925 07:25:30.775159 11139 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad8ac9bf1f5a4c21bcecb65f630798d7 [sys.catalog]: This master's current role is: LEADER
I20260925 07:25:30.775054 11138 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad8ac9bf1f5a4c21bcecb65f630798d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad8ac9bf1f5a4c21bcecb65f630798d7. Latest consensus state: current_term: 1 leader_uuid: "ad8ac9bf1f5a4c21bcecb65f630798d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad8ac9bf1f5a4c21bcecb65f630798d7" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 46115 } } }
I20260925 07:25:30.775333 11138 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad8ac9bf1f5a4c21bcecb65f630798d7 [sys.catalog]: This master's current role is: LEADER
I20260925 07:25:30.775427 11144 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 07:25:30.775918 11144 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 07:25:30.777403 11144 catalog_manager.cc:1347] Generated new cluster ID: a16aa006b67f4a48823f7beb3697828f
I20260925 07:25:30.777457 11144 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 07:25:30.794740 11144 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 07:25:30.795315 11144 catalog_manager.cc:1504] Loading token signing keys...
I20260925 07:25:30.808900 11144 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ad8ac9bf1f5a4c21bcecb65f630798d7: Generated new TSK 0
I20260925 07:25:30.809029 11144 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 07:25:30.823311 11136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:30.820280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42726 (local address 127.5.158.126:46115)
0925 07:25:30.820328 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:30.820330 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:30.820420 (+ 90us) server_negotiation.cc:407] Connection header received
0925 07:25:30.820483 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:30.820484 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:30.820493 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:30.820541 (+ 48us) server_negotiation.cc:226] Negotiated authn=JWT
0925 07:25:30.820667 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:30.821356 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:30.822037 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:30.822207 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:30.822267 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:30.822409 (+ 142us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0925 07:25:30.823134 (+ 725us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34105/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0925 07:25:30.823250 (+ 116us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42726: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34105/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":31}
W20260925 07:25:30.823357 11136 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42726: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34105/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260925 07:25:30.825281 11162 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:25:30.820227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:46115 (local address 127.0.0.1:42726)
0925 07:25:30.820367 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:30.820375 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:30.820434 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:30.820549 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:30.820550 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:30.820552 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0925 07:25:30.820605 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:30.820606 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:30.821442 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:30.821443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:30.821956 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:30.821957 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:30.822267 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:30.822268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:30.822323 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:30.822327 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0925 07:25:30.823441 (+ 1114us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34105/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0925 07:25:30.825196 (+ 1755us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.158.126:46115: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34105/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":1584,"thread_start_us":80,"threads_started":1}
I20260925 07:25:30.825721 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 11066
E20260925 07:25:30.833055 11163 webserver.cc:533] Webserver: sslize failed:
E20260925 07:25:30.833132 11163 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-09-25T07:25:30Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (215 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-25T07:25:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T07:25:30Z Disabled control of system clock
I20260925 07:25:30.868899 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:45429
--webserver_interface=127.5.158.126
--webserver_port=0
--builtin_ntp_servers=127.5.158.84:34611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.158.126:45429
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790321110323796-5753-0/testchainca.cert with env {}
W20260925 07:25:30.938791 11167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:25:30.938973 11167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:25:30.939006 11167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:25:30.939970 11167 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:25:30.940042 11167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:25:30.940061 11167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:25:30.940081 11167 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:25:30.940107 11167 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260925 07:25:30.940138 11167 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260925 07:25:30.940168 11167 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260925 07:25:30.941412 11167 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:34611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790321110323796-5753-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.158.126:45429
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:45429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.158.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790321110323796-5753-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11167
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:25:30.941633 11167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:25:30.941836 11167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:25:30.945791 11175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:30.945837 11173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:30.945796 11172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:30.945968 11167 server_base.cc:1034] running on GCE node
I20260925 07:25:30.946165 11167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:25:30.946388 11167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:25:30.947528 11167 hybrid_clock.cc:648] HybridClock initialized: now 1790321130947518 us; error 33 us; skew 500 ppm
I20260925 07:25:30.951663 11167 webserver.cc:460] Webserver started at http://127.5.158.126:46199/ using document root <none> and password file <none>
I20260925 07:25:30.951865 11167 fs_manager.cc:359] Metadata directory not provided
I20260925 07:25:30.951921 11167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:25:30.952040 11167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:25:30.952978 11167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790321110323796-5753-0/minicluster-data/master-0/data/instance:
uuid: "526224267c5c43198d8a74df1c99ebd3"
format_stamp: "Formatted at 2026-09-25 07:25:30 on dist-test-slave-1drq"
server_key: "7638c3e769875ad6f9b9a790da2a048f"
server_key_iv: "d52cfd2c9dce28ae01ea1f017aba469b"
server_key_version: "encryptionkey@0"
I20260925 07:25:30.953317 11167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790321110323796-5753-0/minicluster-data/master-0/wal/instance:
uuid: "526224267c5c43198d8a74df1c99ebd3"
format_stamp: "Formatted at 2026-09-25 07:25:30 on dist-test-slave-1drq"
server_key: "7638c3e769875ad6f9b9a790da2a048f"
server_key_iv: "d52cfd2c9dce28ae01ea1f017aba469b"
server_key_version: "encryptionkey@0"
I20260925 07:25:30.954355 11167 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 07:25:30.954975 11181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:30.955107 11167 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:25:30.955193 11167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "526224267c5c43198d8a74df1c99ebd3"
format_stamp: "Formatted at 2026-09-25 07:25:30 on dist-test-slave-1drq"
server_key: "7638c3e769875ad6f9b9a790da2a048f"
server_key_iv: "d52cfd2c9dce28ae01ea1f017aba469b"
server_key_version: "encryptionkey@0"
I20260925 07:25:30.955288 11167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:25:30.967057 11167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:25:30.967272 11167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:25:30.967430 11167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:25:30.970695 11167 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.126:45429
I20260925 07:25:30.970741 11233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.126:45429 every 8 connection(s)
I20260925 07:25:30.971014 11167 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:25:30.971778 11234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:25:30.972528 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 11167
I20260925 07:25:30.972605 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790321110323796-5753-0/minicluster-data/master-0/wal/instance
I20260925 07:25:30.972771 5753 external_mini_cluster.cc:1428] Setting key 5c12e9cd43ad70fcd3938dbaf0002ea5
I20260925 07:25:30.974043 11234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 526224267c5c43198d8a74df1c99ebd3: Bootstrap starting.
I20260925 07:25:30.974601 11234 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 526224267c5c43198d8a74df1c99ebd3: Neither blocks nor log segments found. Creating new log.
I20260925 07:25:30.974876 11234 log.cc:824] T 00000000000000000000000000000000 P 526224267c5c43198d8a74df1c99ebd3: Log is configured to *not* fsync() on all Append() calls
I20260925 07:25:30.975486 11234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 526224267c5c43198d8a74df1c99ebd3: No bootstrap required, opened a new log
I20260925 07:25:30.977562 11234 raft_consensus.cc:348] T 00000000000000000000000000000000 P 526224267c5c43198d8a74df1c99ebd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "526224267c5c43198d8a74df1c99ebd3" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 45429 } }
I20260925 07:25:30.977728 11234 raft_consensus.cc:374] T 00000000000000000000000000000000 P 526224267c5c43198d8a74df1c99ebd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:25:30.977764 11234 raft_consensus.cc:729] T 00000000000000000000000000000000 P 526224267c5c43198d8a74df1c99ebd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 526224267c5c43198d8a74df1c99ebd3, State: Initialized, Role: FOLLOWER
I20260925 07:25:30.977885 11234 consensus_queue.cc:260] T 00000000000000000000000000000000 P 526224267c5c43198d8a74df1c99ebd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "526224267c5c43198d8a74df1c99ebd3" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 45429 } }
I20260925 07:25:30.977927 11237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:30.973143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56534 (local address 127.5.158.126:45429)
0925 07:25:30.973324 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:30.973332 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:30.973360 (+ 28us) server_negotiation.cc:407] Connection header received
0925 07:25:30.973783 (+ 423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:30.973791 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:30.973859 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:30.974004 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:30.974156 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:30.975290 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:30.976011 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:30.976572 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:30.976702 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:30.977263 (+ 561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:30.977267 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:30.977268 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:25:30.977482 (+ 214us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:30.977521 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:25:30.977535 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:30.977607 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:30.977656 (+ 49us) server_negotiation.cc:299] Negotiation successful
0925 07:25:30.977734 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":84,"threads_started":1}
I20260925 07:25:30.977975 11234 raft_consensus.cc:388] T 00000000000000000000000000000000 P 526224267c5c43198d8a74df1c99ebd3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 07:25:30.978011 11234 raft_consensus.cc:482] T 00000000000000000000000000000000 P 526224267c5c43198d8a74df1c99ebd3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 07:25:30.978051 11234 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 526224267c5c43198d8a74df1c99ebd3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:25:30.978716 11234 raft_consensus.cc:504] T 00000000000000000000000000000000 P 526224267c5c43198d8a74df1c99ebd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "526224267c5c43198d8a74df1c99ebd3" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 45429 } }
I20260925 07:25:30.978798 5753 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 07:25:30.978811 11234 leader_election.cc:302] T 00000000000000000000000000000000 P 526224267c5c43198d8a74df1c99ebd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 526224267c5c43198d8a74df1c99ebd3; no voters:
I20260925 07:25:30.978942 11234 leader_election.cc:288] T 00000000000000000000000000000000 P 526224267c5c43198d8a74df1c99ebd3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 07:25:30.978982 11239 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 526224267c5c43198d8a74df1c99ebd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:25:30.979101 11239 raft_consensus.cc:686] T 00000000000000000000000000000000 P 526224267c5c43198d8a74df1c99ebd3 [term 1 LEADER]: Becoming Leader. State: Replica: 526224267c5c43198d8a74df1c99ebd3, State: Running, Role: LEADER
I20260925 07:25:30.979156 11234 sys_catalog.cc:564] T 00000000000000000000000000000000 P 526224267c5c43198d8a74df1c99ebd3 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:25:30.979172 11239 consensus_queue.cc:237] T 00000000000000000000000000000000 P 526224267c5c43198d8a74df1c99ebd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "526224267c5c43198d8a74df1c99ebd3" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 45429 } }
I20260925 07:25:30.979451 11239 sys_catalog.cc:455] T 00000000000000000000000000000000 P 526224267c5c43198d8a74df1c99ebd3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 526224267c5c43198d8a74df1c99ebd3. Latest consensus state: current_term: 1 leader_uuid: "526224267c5c43198d8a74df1c99ebd3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "526224267c5c43198d8a74df1c99ebd3" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 45429 } } }
I20260925 07:25:30.979538 11239 sys_catalog.cc:458] T 00000000000000000000000000000000 P 526224267c5c43198d8a74df1c99ebd3 [sys.catalog]: This master's current role is: LEADER
I20260925 07:25:30.979457 11241 sys_catalog.cc:455] T 00000000000000000000000000000000 P 526224267c5c43198d8a74df1c99ebd3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "526224267c5c43198d8a74df1c99ebd3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "526224267c5c43198d8a74df1c99ebd3" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 45429 } } }
I20260925 07:25:30.979995 11241 sys_catalog.cc:458] T 00000000000000000000000000000000 P 526224267c5c43198d8a74df1c99ebd3 [sys.catalog]: This master's current role is: LEADER
I20260925 07:25:30.980214 11245 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 07:25:30.980682 11245 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 07:25:30.982033 11245 catalog_manager.cc:1347] Generated new cluster ID: b1dfa001daa8435b8cb1a1a4ea1f15e8
I20260925 07:25:30.982069 11245 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 07:25:30.996212 11245 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 07:25:30.996604 11245 catalog_manager.cc:1504] Loading token signing keys...
I20260925 07:25:31.000912 11245 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 526224267c5c43198d8a74df1c99ebd3: Generated new TSK 0
I20260925 07:25:31.001049 11245 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 07:25:31.127223 11237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:31.009018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56546 (local address 127.5.158.126:45429)
0925 07:25:31.009063 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:31.009065 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:31.009170 (+ 105us) server_negotiation.cc:407] Connection header received
0925 07:25:31.009222 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:31.009223 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:31.009232 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:31.009296 (+ 64us) server_negotiation.cc:226] Negotiated authn=JWT
0925 07:25:31.009377 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:31.010026 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:31.010685 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:31.010974 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:31.011029 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:31.011160 (+ 131us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0925 07:25:31.127036 (+115876us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0925 07:25:31.127171 (+ 135us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56546: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":29}
W20260925 07:25:31.127280 11237 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56546: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260925 07:25:31.127447 11263 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:25:31.008953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:45429 (local address 127.0.0.1:56546)
0925 07:25:31.009107 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:31.009115 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:31.009175 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:31.009299 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:31.009299 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:31.009301 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0925 07:25:31.009337 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:31.009339 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:31.010085 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:31.010085 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:31.010612 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:31.010613 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:31.011045 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:31.011046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:31.011080 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:31.011084 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0925 07:25:31.127180 (+116096us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0925 07:25:31.127312 (+ 132us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.158.126:45429: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":75,"threads_started":1}
I20260925 07:25:31.127774 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 11167
2026-09-25T07:25:31Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (281 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (78 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (75 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 07:25:31 dist-test-slave-1drq krb5kdc[11275](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 07:25:31 dist-test-slave-1drq krb5kdc[11275](info): set up 2 sockets
Sep 25 07:25:31 dist-test-slave-1drq krb5kdc[11275](info): commencing operation
krb5kdc: starting...
W20260925 07:25:32.284534 5753 mini_kdc.cc:121] Time spent starting KDC: real 0.985s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Sep 25 07:25:32 dist-test-slave-1drq krb5kdc[11275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321132, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T07:25:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T07:25:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
I20260925 07:25:32.383852 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:45657
--webserver_interface=127.5.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:43653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.158.126:45657
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:25:32.452469 11288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:25:32.452618 11288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:25:32.452637 11288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:25:32.453614 11288 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:25:32.453648 11288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:25:32.453660 11288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:25:32.453671 11288 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:25:32.453711 11288 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:25:32.454910 11288 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:43653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.158.126:45657
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:45657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:25:32.455118 11288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:25:32.455302 11288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:25:32.459620 11296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:32.459645 11293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:32.459647 11294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:32.459784 11288 server_base.cc:1034] running on GCE node
I20260925 07:25:32.459964 11288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:25:32.460216 11288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:25:32.461364 11288 hybrid_clock.cc:648] HybridClock initialized: now 1790321132461351 us; error 36 us; skew 500 ppm
Sep 25 07:25:32 dist-test-slave-1drq krb5kdc[11275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321132, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:25:32.463359 11288 init.cc:377] Logged in from keytab as kudu/127.5.158.126@KRBTEST.COM (short username kudu)
I20260925 07:25:32.463755 11288 webserver.cc:460] Webserver started at http://127.5.158.126:44607/ using document root <none> and password file <none>
I20260925 07:25:32.463964 11288 fs_manager.cc:359] Metadata directory not provided
I20260925 07:25:32.464018 11288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:25:32.464147 11288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:25:32.464946 11288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/master-0/data/instance:
uuid: "23ec6d3ebe184fc6a8e9cd0159429c52"
format_stamp: "Formatted at 2026-09-25 07:25:32 on dist-test-slave-1drq"
server_key: "d07d104bc24d4efba00091293910ff29"
server_key_iv: "7d2364a3d8220d769b68be99916b5b24"
server_key_version: "encryptionkey@0"
I20260925 07:25:32.465260 11288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/master-0/wal/instance:
uuid: "23ec6d3ebe184fc6a8e9cd0159429c52"
format_stamp: "Formatted at 2026-09-25 07:25:32 on dist-test-slave-1drq"
server_key: "d07d104bc24d4efba00091293910ff29"
server_key_iv: "7d2364a3d8220d769b68be99916b5b24"
server_key_version: "encryptionkey@0"
I20260925 07:25:32.466298 11288 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 07:25:32.466884 11303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:32.467051 11288 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 07:25:32.467129 11288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "23ec6d3ebe184fc6a8e9cd0159429c52"
format_stamp: "Formatted at 2026-09-25 07:25:32 on dist-test-slave-1drq"
server_key: "d07d104bc24d4efba00091293910ff29"
server_key_iv: "7d2364a3d8220d769b68be99916b5b24"
server_key_version: "encryptionkey@0"
I20260925 07:25:32.467193 11288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:25:32.490672 11288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:25:32.492079 11288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:25:32.492201 11288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:25:32.495348 11288 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.126:45657
I20260925 07:25:32.495409 11355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.126:45657 every 8 connection(s)
I20260925 07:25:32.495671 11288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:25:32.496337 11356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:25:32.497135 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 11288
I20260925 07:25:32.497278 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/master-0/wal/instance
I20260925 07:25:32.497466 5753 external_mini_cluster.cc:1428] Setting key fa573a61e86764d18a2abb03133ad503
I20260925 07:25:32.498711 11356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23ec6d3ebe184fc6a8e9cd0159429c52: Bootstrap starting.
I20260925 07:25:32.499321 11356 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 23ec6d3ebe184fc6a8e9cd0159429c52: Neither blocks nor log segments found. Creating new log.
I20260925 07:25:32.499588 11356 log.cc:824] T 00000000000000000000000000000000 P 23ec6d3ebe184fc6a8e9cd0159429c52: Log is configured to *not* fsync() on all Append() calls
I20260925 07:25:32.500138 11356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23ec6d3ebe184fc6a8e9cd0159429c52: No bootstrap required, opened a new log
I20260925 07:25:32.501626 11356 raft_consensus.cc:348] T 00000000000000000000000000000000 P 23ec6d3ebe184fc6a8e9cd0159429c52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23ec6d3ebe184fc6a8e9cd0159429c52" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 45657 } }
I20260925 07:25:32.501744 11356 raft_consensus.cc:374] T 00000000000000000000000000000000 P 23ec6d3ebe184fc6a8e9cd0159429c52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:25:32.501781 11356 raft_consensus.cc:729] T 00000000000000000000000000000000 P 23ec6d3ebe184fc6a8e9cd0159429c52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23ec6d3ebe184fc6a8e9cd0159429c52, State: Initialized, Role: FOLLOWER
I20260925 07:25:32.501896 11356 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23ec6d3ebe184fc6a8e9cd0159429c52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23ec6d3ebe184fc6a8e9cd0159429c52" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 45657 } }
I20260925 07:25:32.501969 11356 raft_consensus.cc:388] T 00000000000000000000000000000000 P 23ec6d3ebe184fc6a8e9cd0159429c52 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 07:25:32.502007 11356 raft_consensus.cc:482] T 00000000000000000000000000000000 P 23ec6d3ebe184fc6a8e9cd0159429c52 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 07:25:32.502045 11356 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 23ec6d3ebe184fc6a8e9cd0159429c52 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:25:32.502514 11356 raft_consensus.cc:504] T 00000000000000000000000000000000 P 23ec6d3ebe184fc6a8e9cd0159429c52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23ec6d3ebe184fc6a8e9cd0159429c52" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 45657 } }
I20260925 07:25:32.502622 11356 leader_election.cc:302] T 00000000000000000000000000000000 P 23ec6d3ebe184fc6a8e9cd0159429c52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 23ec6d3ebe184fc6a8e9cd0159429c52; no voters:
I20260925 07:25:32.502815 11356 leader_election.cc:288] T 00000000000000000000000000000000 P 23ec6d3ebe184fc6a8e9cd0159429c52 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 07:25:32.502838 11361 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 23ec6d3ebe184fc6a8e9cd0159429c52 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:25:32.502951 11361 raft_consensus.cc:686] T 00000000000000000000000000000000 P 23ec6d3ebe184fc6a8e9cd0159429c52 [term 1 LEADER]: Becoming Leader. State: Replica: 23ec6d3ebe184fc6a8e9cd0159429c52, State: Running, Role: LEADER
I20260925 07:25:32.503038 11356 sys_catalog.cc:564] T 00000000000000000000000000000000 P 23ec6d3ebe184fc6a8e9cd0159429c52 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:25:32.503024 11361 consensus_queue.cc:237] T 00000000000000000000000000000000 P 23ec6d3ebe184fc6a8e9cd0159429c52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23ec6d3ebe184fc6a8e9cd0159429c52" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 45657 } }
Sep 25 07:25:32 dist-test-slave-1drq krb5kdc[11275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321132, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:25:32.503351 11361 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23ec6d3ebe184fc6a8e9cd0159429c52 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23ec6d3ebe184fc6a8e9cd0159429c52. Latest consensus state: current_term: 1 leader_uuid: "23ec6d3ebe184fc6a8e9cd0159429c52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23ec6d3ebe184fc6a8e9cd0159429c52" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 45657 } } }
I20260925 07:25:32.503427 11361 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23ec6d3ebe184fc6a8e9cd0159429c52 [sys.catalog]: This master's current role is: LEADER
I20260925 07:25:32.503361 11363 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23ec6d3ebe184fc6a8e9cd0159429c52 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "23ec6d3ebe184fc6a8e9cd0159429c52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23ec6d3ebe184fc6a8e9cd0159429c52" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 45657 } } }
I20260925 07:25:32.503582 11363 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23ec6d3ebe184fc6a8e9cd0159429c52 [sys.catalog]: This master's current role is: LEADER
I20260925 07:25:32.503638 11366 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 07:25:32.504114 11366 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 07:25:32.505721 11366 catalog_manager.cc:1347] Generated new cluster ID: 44107129cd3e44c3b262555ff5daa8e3
I20260925 07:25:32.505764 11366 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 07:25:32.510500 11359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:32.497908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51054 (local address 127.5.158.126:45657)
0925 07:25:32.498063 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:32.498070 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:32.498092 (+ 22us) server_negotiation.cc:407] Connection header received
0925 07:25:32.498508 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:32.498513 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:32.498550 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:32.498646 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:32.499312 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:32.500134 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:32.500787 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:32.501193 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:32.501293 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:32.503782 (+ 2489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:32.503786 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:32.503786 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:32.503802 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:32.505608 (+ 1806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:32.509380 (+ 3772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:32.509383 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:32.509383 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:32.509483 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:32.509651 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:32.509652 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:32.509652 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:32.509903 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:32.510044 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:32.510122 (+ 78us) server_negotiation.cc:299] Negotiation successful
0925 07:25:32.510328 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":78,"threads_started":1}
I20260925 07:25:32.530782 11366 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 07:25:32.531286 11366 catalog_manager.cc:1504] Loading token signing keys...
I20260925 07:25:32.537866 11366 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 23ec6d3ebe184fc6a8e9cd0159429c52: Generated new TSK 0
I20260925 07:25:32.538028 11366 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.158.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab.
Entry for principal kudu/127.5.158.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab.
I20260925 07:25:32.565888 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.65:0
--local_ip_for_outbound_sockets=127.5.158.65
--webserver_interface=127.5.158.65
--webserver_port=0
--tserver_master_addrs=127.5.158.126:45657
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab
--principal=kudu/127.5.158.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:43653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:25:32.661504 11382 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:25:32.661666 11382 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:25:32.661686 11382 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:25:32.662662 11382 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:25:32.662693 11382 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:25:32.662716 11382 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.65
I20260925 07:25:32.663964 11382 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:43653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab
--principal=kudu/127.5.158.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.158.65
--webserver_port=0
--tserver_master_addrs=127.5.158.126:45657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11382
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.65
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:25:32.664170 11382 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:25:32.664363 11382 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:25:32.668553 11390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:32.668593 11387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:32.668568 11388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:32.668807 11382 server_base.cc:1034] running on GCE node
I20260925 07:25:32.668955 11382 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:25:32.669183 11382 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:25:32.670303 11382 hybrid_clock.cc:648] HybridClock initialized: now 1790321132670289 us; error 30 us; skew 500 ppm
Sep 25 07:25:32 dist-test-slave-1drq krb5kdc[11275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321132, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:25:32.672437 11382 init.cc:377] Logged in from keytab as kudu/127.5.158.65@KRBTEST.COM (short username kudu)
I20260925 07:25:32.672762 11382 webserver.cc:460] Webserver started at http://127.5.158.65:38571/ using document root <none> and password file <none>
I20260925 07:25:32.672952 11382 fs_manager.cc:359] Metadata directory not provided
I20260925 07:25:32.672989 11382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:25:32.673153 11382 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:25:32.673990 11382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/ts-0/data/instance:
uuid: "cbd57f9495244f42bf49130d3ea13a9f"
format_stamp: "Formatted at 2026-09-25 07:25:32 on dist-test-slave-1drq"
server_key: "9ee66a65ad17cab8e97515a940365d3a"
server_key_iv: "2372b3b82ac99d4852546dc565c3e8bb"
server_key_version: "encryptionkey@0"
I20260925 07:25:32.674310 11382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/ts-0/wal/instance:
uuid: "cbd57f9495244f42bf49130d3ea13a9f"
format_stamp: "Formatted at 2026-09-25 07:25:32 on dist-test-slave-1drq"
server_key: "9ee66a65ad17cab8e97515a940365d3a"
server_key_iv: "2372b3b82ac99d4852546dc565c3e8bb"
server_key_version: "encryptionkey@0"
I20260925 07:25:32.675395 11382 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 07:25:32.676059 11397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:32.676237 11382 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:25:32.676313 11382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/ts-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/ts-0/wal
uuid: "cbd57f9495244f42bf49130d3ea13a9f"
format_stamp: "Formatted at 2026-09-25 07:25:32 on dist-test-slave-1drq"
server_key: "9ee66a65ad17cab8e97515a940365d3a"
server_key_iv: "2372b3b82ac99d4852546dc565c3e8bb"
server_key_version: "encryptionkey@0"
I20260925 07:25:32.676362 11382 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:25:32.692747 11382 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:25:32.694314 11382 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:25:32.694408 11382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:25:32.694540 11382 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:25:32.694774 11382 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:25:32.694802 11382 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:32.694819 11382 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:25:32.694855 11382 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:32.699859 11382 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.65:39293
I20260925 07:25:32.699891 11510 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.65:39293 every 8 connection(s)
I20260925 07:25:32.700142 11382 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
Sep 25 07:25:32 dist-test-slave-1drq krb5kdc[11275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321132, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.65@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:25:32.707577 11359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:32.700730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:40963 (local address 127.5.158.126:45657)
0925 07:25:32.700795 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:32.700797 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:32.700906 (+ 109us) server_negotiation.cc:407] Connection header received
0925 07:25:32.701621 (+ 715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:32.701622 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:32.701630 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:32.701674 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:32.702113 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:32.702566 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:32.703530 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:32.703724 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:32.703795 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:32.705566 (+ 1771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:32.705567 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:32.705568 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:32.705574 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:32.706575 (+ 1001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:32.706884 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:32.706884 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:32.706885 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:32.706929 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:32.707083 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:32.707083 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:32.707084 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:32.707188 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:32.707252 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:32.707455 (+ 203us) server_negotiation.cc:299] Negotiation successful
0925 07:25:32.707517 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 07:25:32.707710 11513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:32.700695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:45657 (local address 127.5.158.65:40963)
0925 07:25:32.700817 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:32.700844 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:32.701539 (+ 695us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:32.701715 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:32.701718 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:32.701908 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:32.702057 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:32.702060 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:32.702630 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:32.702630 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:32.703466 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:32.703468 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:32.703800 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:32.703801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:32.703892 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:32.704275 (+ 383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:32.704282 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:32.705478 (+ 1196us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:32.706635 (+ 1157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:32.706640 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:32.706642 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:32.706826 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:32.706965 (+ 139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:32.706965 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:32.706965 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:32.707040 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:32.707257 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:32.707262 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:32.707357 (+ 95us) client_negotiation.cc:769] Sending connection context
0925 07:25:32.707432 (+ 75us) client_negotiation.cc:240] Negotiation successful
0925 07:25:32.707534 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":73,"threads_started":1}
I20260925 07:25:32.708160 11511 heartbeater.cc:347] Connected to a master server at 127.5.158.126:45657
I20260925 07:25:32.708250 11511 heartbeater.cc:464] Registering TS with master...
I20260925 07:25:32.708437 11511 heartbeater.cc:511] Master 127.5.158.126:45657 requested a full tablet report, sending...
I20260925 07:25:32.708869 11320 ts_manager.cc:194] Registered new tserver with Master: cbd57f9495244f42bf49130d3ea13a9f (127.5.158.65:39293)
I20260925 07:25:32.709442 11320 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.65@KRBTEST.COM'} at 127.5.158.65:40963
I20260925 07:25:32.710307 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 11382
I20260925 07:25:32.710413 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/ts-0/wal/instance
I20260925 07:25:32.710584 5753 external_mini_cluster.cc:1428] Setting key b4cc404f873de092c35f3f836a1c7710
WARNING: no policy specified for kudu/127.5.158.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab.
Entry for principal kudu/127.5.158.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab.
I20260925 07:25:32.731956 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.66:0
--local_ip_for_outbound_sockets=127.5.158.66
--webserver_interface=127.5.158.66
--webserver_port=0
--tserver_master_addrs=127.5.158.126:45657
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab
--principal=kudu/127.5.158.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:43653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:25:32.802330 11516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:25:32.802489 11516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:25:32.802510 11516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:25:32.803645 11516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:25:32.803680 11516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:25:32.803723 11516 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.66
I20260925 07:25:32.805023 11516 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:43653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab
--principal=kudu/127.5.158.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.158.66
--webserver_port=0
--tserver_master_addrs=127.5.158.126:45657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11516
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.66
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:25:32.805235 11516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:25:32.805428 11516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 07:25:32.809676 11516 server_base.cc:1034] running on GCE node
W20260925 07:25:32.809653 11521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:32.809638 11524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:32.809653 11522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:32.809959 11516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:25:32.810122 11516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:25:32.811242 11516 hybrid_clock.cc:648] HybridClock initialized: now 1790321132811232 us; error 27 us; skew 500 ppm
Sep 25 07:25:32 dist-test-slave-1drq krb5kdc[11275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321132, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:25:32.813362 11516 init.cc:377] Logged in from keytab as kudu/127.5.158.66@KRBTEST.COM (short username kudu)
I20260925 07:25:32.813723 11516 webserver.cc:460] Webserver started at http://127.5.158.66:46457/ using document root <none> and password file <none>
I20260925 07:25:32.813886 11516 fs_manager.cc:359] Metadata directory not provided
I20260925 07:25:32.813921 11516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:25:32.813997 11516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:25:32.814788 11516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/ts-1/data/instance:
uuid: "5f115e9ebd474c8fad0cbbc6b98b67ac"
format_stamp: "Formatted at 2026-09-25 07:25:32 on dist-test-slave-1drq"
server_key: "f138fbb1503dd01d85e3df41a48a722c"
server_key_iv: "6df08f18b5c75ae1c8cd3b74a653e91c"
server_key_version: "encryptionkey@0"
I20260925 07:25:32.815048 11516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/ts-1/wal/instance:
uuid: "5f115e9ebd474c8fad0cbbc6b98b67ac"
format_stamp: "Formatted at 2026-09-25 07:25:32 on dist-test-slave-1drq"
server_key: "f138fbb1503dd01d85e3df41a48a722c"
server_key_iv: "6df08f18b5c75ae1c8cd3b74a653e91c"
server_key_version: "encryptionkey@0"
I20260925 07:25:32.816146 11516 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 07:25:32.816744 11531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:32.816900 11516 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:32.816982 11516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/ts-1/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/ts-1/wal
uuid: "5f115e9ebd474c8fad0cbbc6b98b67ac"
format_stamp: "Formatted at 2026-09-25 07:25:32 on dist-test-slave-1drq"
server_key: "f138fbb1503dd01d85e3df41a48a722c"
server_key_iv: "6df08f18b5c75ae1c8cd3b74a653e91c"
server_key_version: "encryptionkey@0"
I20260925 07:25:32.817046 11516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:25:32.827194 11516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:25:32.828747 11516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:25:32.828886 11516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:25:32.829124 11516 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:25:32.829402 11516 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:25:32.829447 11516 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:32.829483 11516 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:25:32.829507 11516 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:32.834578 11516 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.66:41299
I20260925 07:25:32.834626 11644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.66:41299 every 8 connection(s)
I20260925 07:25:32.834887 11516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
I20260925 07:25:32.835858 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 11516
I20260925 07:25:32.835938 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/ts-1/wal/instance
I20260925 07:25:32.836122 5753 external_mini_cluster.cc:1428] Setting key db12d19b7a17fa37afc9f56b8ea05806
Sep 25 07:25:32 dist-test-slave-1drq krb5kdc[11275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321132, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.66@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.158.67@KRBTEST.COM; defaulting to no policy
I20260925 07:25:32.843520 11359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:32.835477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:56265 (local address 127.5.158.126:45657)
0925 07:25:32.835542 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:32.835544 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:32.835703 (+ 159us) server_negotiation.cc:407] Connection header received
0925 07:25:32.836720 (+ 1017us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:32.836720 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:32.836729 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:32.836767 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:32.837388 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:32.838148 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:32.839039 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:32.839231 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:32.839297 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:32.841097 (+ 1800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:32.841099 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:32.841099 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:32.841106 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:32.842336 (+ 1230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:32.842705 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:32.842706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:32.842707 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:32.842760 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:32.842980 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:32.842981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:32.842981 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:32.843083 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:32.843158 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:32.843375 (+ 217us) server_negotiation.cc:299] Negotiation successful
0925 07:25:32.843442 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 07:25:32.843693 11647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:32.835443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:45657 (local address 127.5.158.66:56265)
0925 07:25:32.835581 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:32.835625 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:32.836569 (+ 944us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:32.836847 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:32.836850 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:32.837074 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:32.837301 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:32.837304 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:32.838231 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:32.838232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:32.838955 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:32.838957 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:32.839303 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:32.839304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:32.839412 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:32.839841 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:32.839848 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:32.840994 (+ 1146us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:32.842426 (+ 1432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:32.842431 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:32.842433 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:32.842626 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:32.842825 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:32.842825 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:32.842826 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:32.842917 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:32.843151 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:32.843156 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:32.843268 (+ 112us) client_negotiation.cc:769] Sending connection context
0925 07:25:32.843362 (+ 94us) client_negotiation.cc:240] Negotiation successful
0925 07:25:32.843487 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":79,"threads_started":1}
I20260925 07:25:32.844045 11645 heartbeater.cc:347] Connected to a master server at 127.5.158.126:45657
I20260925 07:25:32.844132 11645 heartbeater.cc:464] Registering TS with master...
I20260925 07:25:32.844273 11645 heartbeater.cc:511] Master 127.5.158.126:45657 requested a full tablet report, sending...
I20260925 07:25:32.844640 11320 ts_manager.cc:194] Registered new tserver with Master: 5f115e9ebd474c8fad0cbbc6b98b67ac (127.5.158.66:41299)
I20260925 07:25:32.844995 11320 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.66@KRBTEST.COM'} at 127.5.158.66:56265
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab.
Entry for principal kudu/127.5.158.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab.
I20260925 07:25:32.857883 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.67:0
--local_ip_for_outbound_sockets=127.5.158.67
--webserver_interface=127.5.158.67
--webserver_port=0
--tserver_master_addrs=127.5.158.126:45657
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab
--principal=kudu/127.5.158.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:43653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:25:32.931942 11650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:25:32.932102 11650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:25:32.932123 11650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:25:32.933212 11650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:25:32.933249 11650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:25:32.933275 11650 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.67
I20260925 07:25:32.934635 11650 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:43653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab
--principal=kudu/127.5.158.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.158.67
--webserver_port=0
--tserver_master_addrs=127.5.158.126:45657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11650
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.67
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:25:32.934777 11650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:25:32.934996 11650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:25:32.939560 11656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:32.939617 11655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:32.939738 11650 server_base.cc:1034] running on GCE node
W20260925 07:25:32.939558 11658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:32.940044 11650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:25:32.940266 11650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:25:32.941406 11650 hybrid_clock.cc:648] HybridClock initialized: now 1790321132941391 us; error 31 us; skew 500 ppm
Sep 25 07:25:32 dist-test-slave-1drq krb5kdc[11275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321132, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:25:32.943558 11650 init.cc:377] Logged in from keytab as kudu/127.5.158.67@KRBTEST.COM (short username kudu)
I20260925 07:25:32.943959 11650 webserver.cc:460] Webserver started at http://127.5.158.67:41469/ using document root <none> and password file <none>
I20260925 07:25:32.944154 11650 fs_manager.cc:359] Metadata directory not provided
I20260925 07:25:32.944211 11650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:25:32.944341 11650 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:25:32.945300 11650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/ts-2/data/instance:
uuid: "95e2754c170845c29a86e6f333b55ba3"
format_stamp: "Formatted at 2026-09-25 07:25:32 on dist-test-slave-1drq"
server_key: "1438b9a8ca16310fbac88b2079d76f85"
server_key_iv: "4e86a3a4bfbf4c0a0077e69b9f82a555"
server_key_version: "encryptionkey@0"
I20260925 07:25:32.945627 11650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/ts-2/wal/instance:
uuid: "95e2754c170845c29a86e6f333b55ba3"
format_stamp: "Formatted at 2026-09-25 07:25:32 on dist-test-slave-1drq"
server_key: "1438b9a8ca16310fbac88b2079d76f85"
server_key_iv: "4e86a3a4bfbf4c0a0077e69b9f82a555"
server_key_version: "encryptionkey@0"
I20260925 07:25:32.946781 11650 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 07:25:32.947446 11665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:32.947599 11650 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:25:32.947686 11650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/ts-2/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/ts-2/wal
uuid: "95e2754c170845c29a86e6f333b55ba3"
format_stamp: "Formatted at 2026-09-25 07:25:32 on dist-test-slave-1drq"
server_key: "1438b9a8ca16310fbac88b2079d76f85"
server_key_iv: "4e86a3a4bfbf4c0a0077e69b9f82a555"
server_key_version: "encryptionkey@0"
I20260925 07:25:32.947757 11650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:25:32.968401 11650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:25:32.970077 11650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:25:32.970211 11650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:25:32.970398 11650 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:25:32.970664 11650 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:25:32.970710 11650 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:32.970739 11650 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:25:32.970767 11650 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:32.976176 11650 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.67:32771
I20260925 07:25:32.976209 11778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.67:32771 every 8 connection(s)
I20260925 07:25:32.976478 11650 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
I20260925 07:25:32.981527 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 11650
I20260925 07:25:32.981629 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/minicluster-data/ts-2/wal/instance
I20260925 07:25:32.981854 5753 external_mini_cluster.cc:1428] Setting key 3e129382e03c1b2590e2a10a53fd45af
Sep 25 07:25:32 dist-test-slave-1drq krb5kdc[11275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321132, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.67@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:25:32.984905 11359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:32.977147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:55953 (local address 127.5.158.126:45657)
0925 07:25:32.977185 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:32.977187 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:32.977381 (+ 194us) server_negotiation.cc:407] Connection header received
0925 07:25:32.978324 (+ 943us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:32.978325 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:32.978343 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:32.978401 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:32.978955 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:32.979477 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:32.980382 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:32.980573 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:32.980651 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:32.982889 (+ 2238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:32.982889 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:32.982890 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:32.982896 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:32.983940 (+ 1044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:32.984259 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:32.984259 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:32.984260 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:32.984305 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:32.984485 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:32.984485 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:32.984485 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:32.984574 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:32.984619 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:32.984796 (+ 177us) server_negotiation.cc:299] Negotiation successful
0925 07:25:32.984849 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 07:25:32.985144 11781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:32.977133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:45657 (local address 127.5.158.67:55953)
0925 07:25:32.977279 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:32.977308 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:32.978208 (+ 900us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:32.978460 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:32.978464 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:32.978689 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:32.978859 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:32.978863 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:32.979561 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:32.979562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:32.980295 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:32.980297 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:32.980659 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:32.980660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:32.980778 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:32.981207 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:32.981214 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:32.982816 (+ 1602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:32.984006 (+ 1190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:32.984011 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:32.984012 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:32.984201 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:32.984356 (+ 155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:32.984356 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:32.984356 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:32.984438 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:32.984620 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:32.984624 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:32.984717 (+ 93us) client_negotiation.cc:769] Sending connection context
0925 07:25:32.984786 (+ 69us) client_negotiation.cc:240] Negotiation successful
0925 07:25:32.984884 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":74,"threads_started":1}
I20260925 07:25:32.985517 11779 heartbeater.cc:347] Connected to a master server at 127.5.158.126:45657
I20260925 07:25:32.985592 11779 heartbeater.cc:464] Registering TS with master...
I20260925 07:25:32.985702 11779 heartbeater.cc:511] Master 127.5.158.126:45657 requested a full tablet report, sending...
I20260925 07:25:32.985982 11320 ts_manager.cc:194] Registered new tserver with Master: 95e2754c170845c29a86e6f333b55ba3 (127.5.158.67:32771)
I20260925 07:25:32.986280 11320 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.67@KRBTEST.COM'} at 127.5.158.67:55953
I20260925 07:25:32.993467 5753 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/krb5kdc/test-admin.keytab.
Sep 25 07:25:33 dist-test-slave-1drq krb5kdc[11275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321133, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:25:33.007148 5753 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260925 07:25:33.008579 11786 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:25:33.007900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38126 (local address 127.5.158.65:39293)
0925 07:25:33.008055 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:33.008059 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:33.008067 (+ 8us) server_negotiation.cc:407] Connection header received
0925 07:25:33.008088 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:33.008090 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:33.008103 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:33.008161 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:33.008497 (+ 336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38126: BlockingRecv error: recv got EOF from 127.0.0.1:38126 (error 108)
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":81,"threads_started":1}
W20260925 07:25:33.008637 5753 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/krb5kdc/krb5cc' of type FILE
Sep 25 07:25:33 dist-test-slave-1drq krb5kdc[11275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321133, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:25:33.009539 5753 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 25 07:25:33 dist-test-slave-1drq krb5kdc[11275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321133, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:25:33.010280 5753 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 25 07:25:33 dist-test-slave-1drq krb5kdc[11275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321133, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.158.65@KRBTEST.COM
I20260925 07:25:33.016564 11786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:33.010703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38136 (local address 127.5.158.65:39293)
0925 07:25:33.010762 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:33.010764 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:33.010815 (+ 51us) server_negotiation.cc:407] Connection header received
0925 07:25:33.010860 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:33.010860 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:33.010869 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:33.010914 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:33.011228 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:33.011834 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:33.012442 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:33.012708 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:33.012754 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:33.014472 (+ 1718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:33.014478 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:33.014478 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:33.014483 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:33.015705 (+ 1222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:33.015970 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:33.015972 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:33.015973 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:33.016015 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:33.016134 (+ 119us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:33.016134 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:33.016134 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:33.016221 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:33.016296 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:33.016360 (+ 64us) server_negotiation.cc:299] Negotiation successful
0925 07:25:33.016413 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260925 07:25:33.018126 11786 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:25:33.017618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38152 (local address 127.5.158.65:39293)
0925 07:25:33.017658 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:33.017660 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:33.017698 (+ 38us) server_negotiation.cc:407] Connection header received
0925 07:25:33.017758 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:33.017759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:33.017767 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:33.017808 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:33.018094 (+ 286us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38152: BlockingRecv error: recv got EOF from 127.0.0.1:38152 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260925 07:25:33.018208 5753 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/krb5kdc/krb5cc' of type FILE
Sep 25 07:25:33 dist-test-slave-1drq krb5kdc[11275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321133, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:25:33.019009 5753 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 25 07:25:33 dist-test-slave-1drq krb5kdc[11275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321133, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:25:33.019771 5753 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 25 07:25:33 dist-test-slave-1drq krb5kdc[11275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321133, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.158.65@KRBTEST.COM
I20260925 07:25:33.024981 11786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:33.020150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38164 (local address 127.5.158.65:39293)
0925 07:25:33.020181 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:33.020182 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:33.020213 (+ 31us) server_negotiation.cc:407] Connection header received
0925 07:25:33.020259 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:33.020260 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:33.020265 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:33.020345 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:33.020656 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:33.021055 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:33.021619 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:33.021774 (+ 155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:33.021814 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:33.023315 (+ 1501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:33.023316 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:33.023316 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:33.023321 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:33.024295 (+ 974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:33.024553 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:33.024554 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:33.024554 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:33.024594 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:33.024693 (+ 99us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:33.024693 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:33.024694 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:33.024753 (+ 59us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:33.024806 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:33.024867 (+ 61us) server_negotiation.cc:299] Negotiation successful
0925 07:25:33.024934 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
W20260925 07:25:33.026154 11786 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:25:33.025790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38170 (local address 127.5.158.65:39293)
0925 07:25:33.025821 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:33.025823 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:33.025830 (+ 7us) server_negotiation.cc:407] Connection header received
0925 07:25:33.025885 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:33.025886 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:33.025891 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:33.025923 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:33.026104 (+ 181us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38170: BlockingRecv error: recv got EOF from 127.0.0.1:38170 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260925 07:25:33.026260 5753 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790321110323796-5753-0/krb5kdc/krb5cc' of type FILE
Sep 25 07:25:33 dist-test-slave-1drq krb5kdc[11275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321133, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:25:33.027035 5753 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 25 07:25:33 dist-test-slave-1drq krb5kdc[11275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321133, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:25:33.027808 5753 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 25 07:25:33 dist-test-slave-1drq krb5kdc[11275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321133, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.158.65@KRBTEST.COM
I20260925 07:25:33.033354 11786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:33.028213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38182 (local address 127.5.158.65:39293)
0925 07:25:33.028249 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:33.028250 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:33.028337 (+ 87us) server_negotiation.cc:407] Connection header received
0925 07:25:33.028370 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:33.028371 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:33.028389 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:33.028426 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:33.028724 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:33.029162 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:33.029736 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:33.029906 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:33.029946 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:33.031534 (+ 1588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:33.031535 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:33.031535 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:33.031540 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:33.032586 (+ 1046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:33.032815 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:33.032816 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:33.032816 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:33.032856 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:33.032971 (+ 115us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:33.032972 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:33.032972 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:33.033029 (+ 57us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:33.033077 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:33.033189 (+ 112us) server_negotiation.cc:299] Negotiation successful
0925 07:25:33.033234 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 07:25:33.033854 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 11382
I20260925 07:25:33.038304 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 11516
I20260925 07:25:33.042068 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 11650
I20260925 07:25:33.045836 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 11288
2026-09-25T07:25:33Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1765 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 07:25:33 dist-test-slave-1drq krb5kdc[11799](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 07:25:33 dist-test-slave-1drq krb5kdc[11799](info): set up 2 sockets
Sep 25 07:25:33 dist-test-slave-1drq krb5kdc[11799](info): commencing operation
krb5kdc: starting...
W20260925 07:25:33.981010 5753 mini_kdc.cc:121] Time spent starting KDC: real 0.916s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Sep 25 07:25:34 dist-test-slave-1drq krb5kdc[11799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321134, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T07:25:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T07:25:34Z Disabled control of system clock
WARNING: no policy specified for oryx/127.5.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/oryx_127.5.158.126.keytab.
Entry for principal oryx/127.5.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/oryx_127.5.158.126.keytab.
I20260925 07:25:34.084679 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:37781
--webserver_interface=127.5.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/oryx_127.5.158.126.keytab
--principal=oryx/127.5.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:45003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.158.126:37781
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:25:34.155853 11812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:25:34.156065 11812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:25:34.156100 11812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:25:34.157219 11812 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:25:34.157280 11812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:25:34.157305 11812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:25:34.157325 11812 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:25:34.157351 11812 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:25:34.158736 11812 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:45003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.158.126:37781
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/oryx_127.5.158.126.keytab
--principal=oryx/127.5.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:37781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11812
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:25:34.158980 11812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:25:34.159235 11812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:25:34.163777 11818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:34.163797 11817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:34.163784 11820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:34.164021 11812 server_base.cc:1034] running on GCE node
I20260925 07:25:34.164268 11812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:25:34.164491 11812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:25:34.165622 11812 hybrid_clock.cc:648] HybridClock initialized: now 1790321134165609 us; error 27 us; skew 500 ppm
Sep 25 07:25:34 dist-test-slave-1drq krb5kdc[11799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321134, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:25:34.167964 11812 init.cc:377] Logged in from keytab as oryx/127.5.158.126@KRBTEST.COM (short username oryx)
I20260925 07:25:34.168362 11812 webserver.cc:460] Webserver started at http://127.5.158.126:40303/ using document root <none> and password file <none>
I20260925 07:25:34.168545 11812 fs_manager.cc:359] Metadata directory not provided
I20260925 07:25:34.168601 11812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:25:34.168716 11812 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:25:34.169646 11812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-0/data/instance:
uuid: "ea66681bd57f42b0a9db2845ec74d84d"
format_stamp: "Formatted at 2026-09-25 07:25:34 on dist-test-slave-1drq"
server_key: "ca02920758933cd0aa37cf2eb7885ebd"
server_key_iv: "81f21a3306514518f1e520df26ec3f61"
server_key_version: "encryptionkey@0"
I20260925 07:25:34.169962 11812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-0/wal/instance:
uuid: "ea66681bd57f42b0a9db2845ec74d84d"
format_stamp: "Formatted at 2026-09-25 07:25:34 on dist-test-slave-1drq"
server_key: "ca02920758933cd0aa37cf2eb7885ebd"
server_key_iv: "81f21a3306514518f1e520df26ec3f61"
server_key_version: "encryptionkey@0"
I20260925 07:25:34.171100 11812 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 07:25:34.171752 11827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:34.171929 11812 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:25:34.172025 11812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "ea66681bd57f42b0a9db2845ec74d84d"
format_stamp: "Formatted at 2026-09-25 07:25:34 on dist-test-slave-1drq"
server_key: "ca02920758933cd0aa37cf2eb7885ebd"
server_key_iv: "81f21a3306514518f1e520df26ec3f61"
server_key_version: "encryptionkey@0"
I20260925 07:25:34.172091 11812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:25:34.182722 11812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:25:34.184190 11812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:25:34.184340 11812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:25:34.187919 11812 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.126:37781
I20260925 07:25:34.187963 11879 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.126:37781 every 8 connection(s)
I20260925 07:25:34.188277 11812 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:25:34.189193 11880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:25:34.191025 11880 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea66681bd57f42b0a9db2845ec74d84d: Bootstrap starting.
I20260925 07:25:34.191464 11880 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea66681bd57f42b0a9db2845ec74d84d: Neither blocks nor log segments found. Creating new log.
I20260925 07:25:34.191705 11880 log.cc:824] T 00000000000000000000000000000000 P ea66681bd57f42b0a9db2845ec74d84d: Log is configured to *not* fsync() on all Append() calls
I20260925 07:25:34.192283 11880 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea66681bd57f42b0a9db2845ec74d84d: No bootstrap required, opened a new log
I20260925 07:25:34.193787 11880 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea66681bd57f42b0a9db2845ec74d84d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea66681bd57f42b0a9db2845ec74d84d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37781 } }
I20260925 07:25:34.193903 11880 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea66681bd57f42b0a9db2845ec74d84d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:25:34.193923 11880 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea66681bd57f42b0a9db2845ec74d84d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea66681bd57f42b0a9db2845ec74d84d, State: Initialized, Role: FOLLOWER
I20260925 07:25:34.194015 11880 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea66681bd57f42b0a9db2845ec74d84d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea66681bd57f42b0a9db2845ec74d84d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37781 } }
I20260925 07:25:34.194058 11880 raft_consensus.cc:388] T 00000000000000000000000000000000 P ea66681bd57f42b0a9db2845ec74d84d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 07:25:34.194096 11880 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea66681bd57f42b0a9db2845ec74d84d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 07:25:34.194128 11880 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea66681bd57f42b0a9db2845ec74d84d [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:25:34.194835 11880 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea66681bd57f42b0a9db2845ec74d84d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea66681bd57f42b0a9db2845ec74d84d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37781 } }
I20260925 07:25:34.194932 11880 leader_election.cc:302] T 00000000000000000000000000000000 P ea66681bd57f42b0a9db2845ec74d84d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ea66681bd57f42b0a9db2845ec74d84d; no voters:
I20260925 07:25:34.195046 11880 leader_election.cc:288] T 00000000000000000000000000000000 P ea66681bd57f42b0a9db2845ec74d84d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 07:25:34.195111 11883 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea66681bd57f42b0a9db2845ec74d84d [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:25:34.195250 11883 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea66681bd57f42b0a9db2845ec74d84d [term 1 LEADER]: Becoming Leader. State: Replica: ea66681bd57f42b0a9db2845ec74d84d, State: Running, Role: LEADER
I20260925 07:25:34.195336 11880 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea66681bd57f42b0a9db2845ec74d84d [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:25:34.195341 11883 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea66681bd57f42b0a9db2845ec74d84d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea66681bd57f42b0a9db2845ec74d84d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37781 } }
I20260925 07:25:34.195700 11884 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea66681bd57f42b0a9db2845ec74d84d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ea66681bd57f42b0a9db2845ec74d84d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea66681bd57f42b0a9db2845ec74d84d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37781 } } }
I20260925 07:25:34.195710 11885 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea66681bd57f42b0a9db2845ec74d84d [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea66681bd57f42b0a9db2845ec74d84d. Latest consensus state: current_term: 1 leader_uuid: "ea66681bd57f42b0a9db2845ec74d84d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea66681bd57f42b0a9db2845ec74d84d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37781 } } }
I20260925 07:25:34.195820 11884 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea66681bd57f42b0a9db2845ec74d84d [sys.catalog]: This master's current role is: LEADER
I20260925 07:25:34.195875 11885 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea66681bd57f42b0a9db2845ec74d84d [sys.catalog]: This master's current role is: LEADER
I20260925 07:25:34.196290 11891 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 07:25:34.196787 11891 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 07:25:34.198231 11891 catalog_manager.cc:1347] Generated new cluster ID: c10bd3f7239e4d6aaf544a03c0ac2c01
I20260925 07:25:34.198275 11891 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 07:25:34.198261 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 11812
I20260925 07:25:34.198355 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-0/wal/instance
I20260925 07:25:34.198537 5753 external_mini_cluster.cc:1428] Setting key e028b82d72b916fa801de5049da27497
Sep 25 07:25:34 dist-test-slave-1drq krb5kdc[11799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321134, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM
I20260925 07:25:34.207422 11903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:34.198954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59388 (local address 127.5.158.126:37781)
0925 07:25:34.199100 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:34.199109 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:34.199150 (+ 41us) server_negotiation.cc:407] Connection header received
0925 07:25:34.199265 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:34.199270 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:34.199306 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:34.199422 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:34.199977 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:34.200883 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:34.201665 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:34.202069 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:34.202186 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:34.204328 (+ 2142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:34.204332 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:34.204332 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:34.204342 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:34.205932 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:34.206293 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:34.206296 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:34.206297 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:34.206386 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:34.206581 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:34.206582 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:34.206582 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:34.206808 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:34.206967 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:34.207063 (+ 96us) server_negotiation.cc:299] Negotiation successful
0925 07:25:34.207202 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":63,"threads_started":1}
I20260925 07:25:34.216408 11891 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 07:25:34.216872 11891 catalog_manager.cc:1504] Loading token signing keys...
I20260925 07:25:34.219776 11891 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea66681bd57f42b0a9db2845ec74d84d: Generated new TSK 0
I20260925 07:25:34.219911 11891 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.5.158.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.158.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.158.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/oryx_127.5.158.65.keytab.
Entry for principal oryx/127.5.158.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/oryx_127.5.158.65.keytab.
I20260925 07:25:34.245810 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.65:0
--local_ip_for_outbound_sockets=127.5.158.65
--webserver_interface=127.5.158.65
--webserver_port=0
--tserver_master_addrs=127.5.158.126:37781
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/oryx_127.5.158.65.keytab
--principal=oryx/127.5.158.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:45003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:25:34.316447 11906 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:25:34.316614 11906 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:25:34.316634 11906 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:25:34.317775 11906 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:25:34.317813 11906 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:25:34.317862 11906 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.65
I20260925 07:25:34.319206 11906 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:45003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/oryx_127.5.158.65.keytab
--principal=oryx/127.5.158.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.158.65
--webserver_port=0
--tserver_master_addrs=127.5.158.126:37781
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11906
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.65
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:25:34.319408 11906 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:25:34.319586 11906 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:25:34.323809 11914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:34.323743 11911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:34.323742 11912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:34.324000 11906 server_base.cc:1034] running on GCE node
I20260925 07:25:34.324205 11906 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:25:34.324414 11906 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:25:34.325539 11906 hybrid_clock.cc:648] HybridClock initialized: now 1790321134325522 us; error 31 us; skew 500 ppm
Sep 25 07:25:34 dist-test-slave-1drq krb5kdc[11799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321134, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.158.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:25:34.327786 11906 init.cc:377] Logged in from keytab as oryx/127.5.158.65@KRBTEST.COM (short username oryx)
I20260925 07:25:34.328197 11906 webserver.cc:460] Webserver started at http://127.5.158.65:41183/ using document root <none> and password file <none>
I20260925 07:25:34.328373 11906 fs_manager.cc:359] Metadata directory not provided
I20260925 07:25:34.328408 11906 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:25:34.328482 11906 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:25:34.329420 11906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-0/data/instance:
uuid: "633f6da4f11c4910becf8aa67dc147c9"
format_stamp: "Formatted at 2026-09-25 07:25:34 on dist-test-slave-1drq"
server_key: "76fc65242c36c6d69c472fdcc93351c0"
server_key_iv: "7cf0dfedeffbfb7b4973467cfa0123ed"
server_key_version: "encryptionkey@0"
I20260925 07:25:34.329694 11906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-0/wal/instance:
uuid: "633f6da4f11c4910becf8aa67dc147c9"
format_stamp: "Formatted at 2026-09-25 07:25:34 on dist-test-slave-1drq"
server_key: "76fc65242c36c6d69c472fdcc93351c0"
server_key_iv: "7cf0dfedeffbfb7b4973467cfa0123ed"
server_key_version: "encryptionkey@0"
I20260925 07:25:34.330819 11906 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 07:25:34.331424 11921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:34.331570 11906 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:25:34.331641 11906 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-0/wal
uuid: "633f6da4f11c4910becf8aa67dc147c9"
format_stamp: "Formatted at 2026-09-25 07:25:34 on dist-test-slave-1drq"
server_key: "76fc65242c36c6d69c472fdcc93351c0"
server_key_iv: "7cf0dfedeffbfb7b4973467cfa0123ed"
server_key_version: "encryptionkey@0"
I20260925 07:25:34.331688 11906 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:25:34.343294 11906 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:25:34.344761 11906 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:25:34.344861 11906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:25:34.345000 11906 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:25:34.345259 11906 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:25:34.345290 11906 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:34.345310 11906 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:25:34.345321 11906 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:34.350656 11906 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.65:41917
I20260925 07:25:34.350734 12034 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.65:41917 every 8 connection(s)
I20260925 07:25:34.351023 11906 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
Sep 25 07:25:34 dist-test-slave-1drq krb5kdc[11799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321134, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.158.65@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM
I20260925 07:25:34.359048 11903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:34.351709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:34561 (local address 127.5.158.126:37781)
0925 07:25:34.351751 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:34.351754 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:34.351971 (+ 217us) server_negotiation.cc:407] Connection header received
0925 07:25:34.352369 (+ 398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:34.352370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:34.352382 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:34.352428 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:34.353251 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:34.353671 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:34.354506 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:34.354668 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:34.354712 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:34.356607 (+ 1895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:34.356608 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:34.356608 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:34.356612 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:34.357860 (+ 1248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:34.358193 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:34.358194 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:34.358194 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:34.358252 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:34.358450 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:34.358450 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:34.358451 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:34.358531 (+ 80us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:34.358599 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:34.358881 (+ 282us) server_negotiation.cc:299] Negotiation successful
0925 07:25:34.358962 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 07:25:34.359207 12037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:34.351670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.65:34561)
0925 07:25:34.351847 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:34.351888 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:34.352297 (+ 409us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:34.352471 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:34.352474 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:34.352948 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:34.353171 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:34.353176 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:34.353730 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:34.353731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:34.354444 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:34.354445 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:34.354728 (+ 283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:34.354728 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:34.354819 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:34.355262 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:34.355269 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:34.356521 (+ 1252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:34.357924 (+ 1403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:34.357930 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:34.357932 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:34.358133 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:34.358300 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:34.358300 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:34.358300 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:34.358405 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:34.358594 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:34.358601 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:34.358732 (+ 131us) client_negotiation.cc:769] Sending connection context
0925 07:25:34.358858 (+ 126us) client_negotiation.cc:240] Negotiation successful
0925 07:25:34.358998 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":73,"threads_started":1}
I20260925 07:25:34.359541 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 11906
I20260925 07:25:34.359647 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-0/wal/instance
I20260925 07:25:34.359781 12035 heartbeater.cc:347] Connected to a master server at 127.5.158.126:37781
I20260925 07:25:34.359889 5753 external_mini_cluster.cc:1428] Setting key 5cd64f0e061cecfcb66d05f6e3197bea
I20260925 07:25:34.359961 12035 heartbeater.cc:464] Registering TS with master...
I20260925 07:25:34.360176 12035 heartbeater.cc:511] Master 127.5.158.126:37781 requested a full tablet report, sending...
I20260925 07:25:34.360659 11844 ts_manager.cc:194] Registered new tserver with Master: 633f6da4f11c4910becf8aa67dc147c9 (127.5.158.65:41917)
I20260925 07:25:34.361466 11844 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.158.65@KRBTEST.COM'} at 127.5.158.65:34561
WARNING: no policy specified for oryx/127.5.158.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.158.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.158.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/oryx_127.5.158.66.keytab.
Entry for principal oryx/127.5.158.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/oryx_127.5.158.66.keytab.
I20260925 07:25:34.381701 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.66:0
--local_ip_for_outbound_sockets=127.5.158.66
--webserver_interface=127.5.158.66
--webserver_port=0
--tserver_master_addrs=127.5.158.126:37781
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/oryx_127.5.158.66.keytab
--principal=oryx/127.5.158.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:45003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:25:34.453600 12040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:25:34.453748 12040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:25:34.453773 12040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:25:34.454656 12040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:25:34.454684 12040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:25:34.454704 12040 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.66
I20260925 07:25:34.455936 12040 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:45003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/oryx_127.5.158.66.keytab
--principal=oryx/127.5.158.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.158.66
--webserver_port=0
--tserver_master_addrs=127.5.158.126:37781
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12040
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.66
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:25:34.456086 12040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:25:34.456225 12040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:25:34.460527 12045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:34.460646 12046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:34.460629 12040 server_base.cc:1034] running on GCE node
W20260925 07:25:34.460556 12048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:34.460897 12040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:25:34.461126 12040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:25:34.462263 12040 hybrid_clock.cc:648] HybridClock initialized: now 1790321134462247 us; error 33 us; skew 500 ppm
Sep 25 07:25:34 dist-test-slave-1drq krb5kdc[11799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321134, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.158.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:25:34.464457 12040 init.cc:377] Logged in from keytab as oryx/127.5.158.66@KRBTEST.COM (short username oryx)
I20260925 07:25:34.464818 12040 webserver.cc:460] Webserver started at http://127.5.158.66:34853/ using document root <none> and password file <none>
I20260925 07:25:34.465034 12040 fs_manager.cc:359] Metadata directory not provided
I20260925 07:25:34.465075 12040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:25:34.465256 12040 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:25:34.466110 12040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-1/data/instance:
uuid: "2bcb44a2433144509be8441477797a48"
format_stamp: "Formatted at 2026-09-25 07:25:34 on dist-test-slave-1drq"
server_key: "b7512700d5dab35d6ecefbb430246152"
server_key_iv: "abca6e360c20d137c88d7c1ca73f7028"
server_key_version: "encryptionkey@0"
I20260925 07:25:34.466404 12040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-1/wal/instance:
uuid: "2bcb44a2433144509be8441477797a48"
format_stamp: "Formatted at 2026-09-25 07:25:34 on dist-test-slave-1drq"
server_key: "b7512700d5dab35d6ecefbb430246152"
server_key_iv: "abca6e360c20d137c88d7c1ca73f7028"
server_key_version: "encryptionkey@0"
I20260925 07:25:34.467571 12040 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 07:25:34.468218 12055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:34.468390 12040 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:34.468462 12040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-1/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-1/wal
uuid: "2bcb44a2433144509be8441477797a48"
format_stamp: "Formatted at 2026-09-25 07:25:34 on dist-test-slave-1drq"
server_key: "b7512700d5dab35d6ecefbb430246152"
server_key_iv: "abca6e360c20d137c88d7c1ca73f7028"
server_key_version: "encryptionkey@0"
I20260925 07:25:34.468506 12040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:25:34.479146 12040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:25:34.480988 12040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:25:34.481117 12040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:25:34.481289 12040 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:25:34.481545 12040 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:25:34.481575 12040 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:34.481593 12040 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:25:34.481606 12040 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:34.486649 12040 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.66:32935
I20260925 07:25:34.486722 12168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.66:32935 every 8 connection(s)
I20260925 07:25:34.486950 12040 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
Sep 25 07:25:34 dist-test-slave-1drq krb5kdc[11799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321134, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.158.66@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM
I20260925 07:25:34.495352 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 12040
I20260925 07:25:34.495494 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-1/wal/instance
I20260925 07:25:34.495666 5753 external_mini_cluster.cc:1428] Setting key 9d7b0d2afff0997744e4d19e1a0e4b78
I20260925 07:25:34.495769 11903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:34.487591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:40543 (local address 127.5.158.126:37781)
0925 07:25:34.487649 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:34.487652 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:34.487863 (+ 211us) server_negotiation.cc:407] Connection header received
0925 07:25:34.488293 (+ 430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:34.488295 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:34.488312 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:34.488402 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:34.489235 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:34.490024 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:34.490931 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:34.491231 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:34.491309 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:34.493269 (+ 1960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:34.493270 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:34.493270 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:34.493277 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:34.494384 (+ 1107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:34.494764 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:34.494765 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:34.494765 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:34.494820 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:34.495071 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:34.495072 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:34.495072 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:34.495185 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:34.495261 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:34.495554 (+ 293us) server_negotiation.cc:299] Negotiation successful
0925 07:25:34.495651 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260925 07:25:34.495854 12171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:34.487589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.66:40543)
0925 07:25:34.487767 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:34.487798 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:34.488182 (+ 384us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:34.488442 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:34.488445 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:34.488928 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:34.489147 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:34.489152 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:34.490119 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:34.490120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:34.490850 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:34.490852 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:34.491328 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:34.491329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:34.491418 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:34.491822 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:34.491828 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:34.493164 (+ 1336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:34.494446 (+ 1282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:34.494451 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:34.494452 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:34.494682 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:34.494883 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:34.494884 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:34.494884 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:34.495003 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:34.495247 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:34.495253 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:34.495403 (+ 150us) client_negotiation.cc:769] Sending connection context
0925 07:25:34.495522 (+ 119us) client_negotiation.cc:240] Negotiation successful
0925 07:25:34.495663 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":99,"threads_started":1}
I20260925 07:25:34.496206 12169 heartbeater.cc:347] Connected to a master server at 127.5.158.126:37781
I20260925 07:25:34.496289 12169 heartbeater.cc:464] Registering TS with master...
I20260925 07:25:34.496450 12169 heartbeater.cc:511] Master 127.5.158.126:37781 requested a full tablet report, sending...
I20260925 07:25:34.496791 11844 ts_manager.cc:194] Registered new tserver with Master: 2bcb44a2433144509be8441477797a48 (127.5.158.66:32935)
I20260925 07:25:34.497335 11844 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.158.66@KRBTEST.COM'} at 127.5.158.66:40543
WARNING: no policy specified for oryx/127.5.158.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.158.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.158.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/oryx_127.5.158.67.keytab.
Entry for principal oryx/127.5.158.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/oryx_127.5.158.67.keytab.
I20260925 07:25:34.518976 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.67:0
--local_ip_for_outbound_sockets=127.5.158.67
--webserver_interface=127.5.158.67
--webserver_port=0
--tserver_master_addrs=127.5.158.126:37781
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/oryx_127.5.158.67.keytab
--principal=oryx/127.5.158.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:45003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:25:34.592088 12174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:25:34.592273 12174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:25:34.592304 12174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:25:34.593315 12174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:25:34.593364 12174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:25:34.593400 12174 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.67
I20260925 07:25:34.594676 12174 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:45003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/oryx_127.5.158.67.keytab
--principal=oryx/127.5.158.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.158.67
--webserver_port=0
--tserver_master_addrs=127.5.158.126:37781
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12174
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.67
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:25:34.594903 12174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:25:34.595130 12174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:25:34.599555 12180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:34.599555 12179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:34.599589 12182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:34.599696 12174 server_base.cc:1034] running on GCE node
I20260925 07:25:34.599979 12174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:25:34.600181 12174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:25:34.601323 12174 hybrid_clock.cc:648] HybridClock initialized: now 1790321134601296 us; error 38 us; skew 500 ppm
Sep 25 07:25:34 dist-test-slave-1drq krb5kdc[11799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321134, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.158.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:25:34.603392 12174 init.cc:377] Logged in from keytab as oryx/127.5.158.67@KRBTEST.COM (short username oryx)
I20260925 07:25:34.603768 12174 webserver.cc:460] Webserver started at http://127.5.158.67:35851/ using document root <none> and password file <none>
I20260925 07:25:34.603998 12174 fs_manager.cc:359] Metadata directory not provided
I20260925 07:25:34.604053 12174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:25:34.604179 12174 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:25:34.605141 12174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-2/data/instance:
uuid: "470984d5f90d4994b15fdb6d0d13e6c6"
format_stamp: "Formatted at 2026-09-25 07:25:34 on dist-test-slave-1drq"
server_key: "d3b9bbae6d4630f47eb64a090c7c2c49"
server_key_iv: "64b1e487e0803b4133a689d1411c4804"
server_key_version: "encryptionkey@0"
I20260925 07:25:34.605468 12174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-2/wal/instance:
uuid: "470984d5f90d4994b15fdb6d0d13e6c6"
format_stamp: "Formatted at 2026-09-25 07:25:34 on dist-test-slave-1drq"
server_key: "d3b9bbae6d4630f47eb64a090c7c2c49"
server_key_iv: "64b1e487e0803b4133a689d1411c4804"
server_key_version: "encryptionkey@0"
I20260925 07:25:34.606580 12174 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 07:25:34.607237 12189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:34.607393 12174 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:25:34.607478 12174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-2/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-2/wal
uuid: "470984d5f90d4994b15fdb6d0d13e6c6"
format_stamp: "Formatted at 2026-09-25 07:25:34 on dist-test-slave-1drq"
server_key: "d3b9bbae6d4630f47eb64a090c7c2c49"
server_key_iv: "64b1e487e0803b4133a689d1411c4804"
server_key_version: "encryptionkey@0"
I20260925 07:25:34.607554 12174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:25:34.623008 12174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:25:34.624493 12174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:25:34.624626 12174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:25:34.624805 12174 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:25:34.625051 12174 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:25:34.625113 12174 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:34.625154 12174 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:25:34.625180 12174 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:34.630380 12174 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.67:44005
I20260925 07:25:34.630445 12302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.67:44005 every 8 connection(s)
I20260925 07:25:34.630692 12174 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
I20260925 07:25:34.632597 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 12174
I20260925 07:25:34.632669 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-2/wal/instance
I20260925 07:25:34.632859 5753 external_mini_cluster.cc:1428] Setting key f9939184476c1ade549c602326560663
Sep 25 07:25:34 dist-test-slave-1drq krb5kdc[11799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321134, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.158.67@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM
I20260925 07:25:34.639012 11903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:34.631334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:52367 (local address 127.5.158.126:37781)
0925 07:25:34.631385 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:34.631387 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:34.631543 (+ 156us) server_negotiation.cc:407] Connection header received
0925 07:25:34.631930 (+ 387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:34.631931 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:34.631940 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:34.631985 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:34.632825 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:34.633572 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:34.634557 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:34.634716 (+ 159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:34.634776 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:34.636936 (+ 2160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:34.636937 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:34.636937 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:34.636943 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:34.638086 (+ 1143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:34.638421 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:34.638421 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:34.638422 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:34.638463 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:34.638622 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:34.638623 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:34.638623 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:34.638703 (+ 80us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:34.638745 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:34.638909 (+ 164us) server_negotiation.cc:299] Negotiation successful
0925 07:25:34.638959 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260925 07:25:34.639197 12305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:34.631308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.67:52367)
0925 07:25:34.631441 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:34.631469 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:34.631857 (+ 388us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:34.632060 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:34.632063 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:34.632536 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:34.632744 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:34.632749 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:34.633644 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:34.633646 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:34.634490 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:34.634492 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:34.634778 (+ 286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:34.634778 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:34.634856 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:34.635331 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:34.635341 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:34.636864 (+ 1523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:34.638160 (+ 1296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:34.638165 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:34.638167 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:34.638365 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:34.638499 (+ 134us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:34.638499 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:34.638499 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:34.638578 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:34.638736 (+ 158us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:34.638738 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:34.638824 (+ 86us) client_negotiation.cc:769] Sending connection context
0925 07:25:34.638887 (+ 63us) client_negotiation.cc:240] Negotiation successful
0925 07:25:34.638982 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":75,"threads_started":1}
I20260925 07:25:34.639602 12303 heartbeater.cc:347] Connected to a master server at 127.5.158.126:37781
I20260925 07:25:34.639678 12303 heartbeater.cc:464] Registering TS with master...
I20260925 07:25:34.639816 12303 heartbeater.cc:511] Master 127.5.158.126:37781 requested a full tablet report, sending...
I20260925 07:25:34.640136 11844 ts_manager.cc:194] Registered new tserver with Master: 470984d5f90d4994b15fdb6d0d13e6c6 (127.5.158.67:44005)
I20260925 07:25:34.640407 11844 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.158.67@KRBTEST.COM'} at 127.5.158.67:52367
I20260925 07:25:34.644279 5753 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.5.158.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.158.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.158.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/oryx_127.5.158.125.keytab.
Entry for principal oryx/127.5.158.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/oryx_127.5.158.125.keytab.
I20260925 07:25:34.665774 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.125:44439
--webserver_interface=127.5.158.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/oryx_127.5.158.125.keytab
--principal=oryx/127.5.158.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:45003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.158.126:37781,127.5.158.125:44439
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:25:34.738441 12308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:25:34.738615 12308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:25:34.738633 12308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:25:34.739727 12308 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:25:34.739761 12308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:25:34.739774 12308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:25:34.739830 12308 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:25:34.739862 12308 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:25:34.741190 12308 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:45003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.158.126:37781,127.5.158.125:44439
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/oryx_127.5.158.125.keytab
--principal=oryx/127.5.158.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.125:44439
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.158.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12308
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
W20260925 07:25:34.741285 12308 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.5.158.126:37781,127.5.158.125:44439'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260925 07:25:34.741396 12308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:25:34.741572 12308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:25:34.745739 12314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:34.745831 12316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:34.745746 12313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:34.745860 12308 server_base.cc:1034] running on GCE node
I20260925 07:25:34.746068 12308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:25:34.746330 12308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:25:34.747478 12308 hybrid_clock.cc:648] HybridClock initialized: now 1790321134747470 us; error 28 us; skew 500 ppm
Sep 25 07:25:34 dist-test-slave-1drq krb5kdc[11799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321134, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.158.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:25:34.749845 12308 init.cc:377] Logged in from keytab as oryx/127.5.158.125@KRBTEST.COM (short username oryx)
I20260925 07:25:34.750233 12308 webserver.cc:460] Webserver started at http://127.5.158.125:39737/ using document root <none> and password file <none>
I20260925 07:25:34.750448 12308 fs_manager.cc:359] Metadata directory not provided
I20260925 07:25:34.750517 12308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:25:34.750638 12308 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:25:34.751487 12308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-1/data/instance:
uuid: "87d6c9fb88664a3aa4245a34c108f564"
format_stamp: "Formatted at 2026-09-25 07:25:34 on dist-test-slave-1drq"
server_key: "4f406b533db55a9d62f5a97656932132"
server_key_iv: "fb4bc136b3a5518fe6be0fd6826bd2ed"
server_key_version: "encryptionkey@0"
I20260925 07:25:34.751847 12308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-1/wal/instance:
uuid: "87d6c9fb88664a3aa4245a34c108f564"
format_stamp: "Formatted at 2026-09-25 07:25:34 on dist-test-slave-1drq"
server_key: "4f406b533db55a9d62f5a97656932132"
server_key_iv: "fb4bc136b3a5518fe6be0fd6826bd2ed"
server_key_version: "encryptionkey@0"
I20260925 07:25:34.753125 12308 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 07:25:34.753815 12323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:34.753973 12308 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 07:25:34.754060 12308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-1/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-1/wal
uuid: "87d6c9fb88664a3aa4245a34c108f564"
format_stamp: "Formatted at 2026-09-25 07:25:34 on dist-test-slave-1drq"
server_key: "4f406b533db55a9d62f5a97656932132"
server_key_iv: "fb4bc136b3a5518fe6be0fd6826bd2ed"
server_key_version: "encryptionkey@0"
I20260925 07:25:34.754134 12308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:25:34.766247 12308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:25:34.767882 12308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:25:34.768030 12308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:25:34.771716 12308 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.125:44439
I20260925 07:25:34.771750 12375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.125:44439 every 8 connection(s)
I20260925 07:25:34.772073 12308 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-1/data/info.pb
I20260925 07:25:34.772915 12376 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:25:34.774296 12376 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37781 } has no permanent_uuid. Determining permanent_uuid...
I20260925 07:25:34.779402 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 12308
Sep 25 07:25:34 dist-test-slave-1drq krb5kdc[11799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321134, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.158.125@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM
I20260925 07:25:34.782835 11903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:34.774885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59394 (local address 127.5.158.126:37781)
0925 07:25:34.774939 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:34.774941 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:34.775122 (+ 181us) server_negotiation.cc:407] Connection header received
0925 07:25:34.775590 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:34.775591 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:34.775603 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:34.775651 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:34.776183 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:34.776655 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:34.777691 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:34.777912 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:34.777994 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:34.780088 (+ 2094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:34.780089 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:34.780090 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:34.780097 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:34.781548 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:34.781889 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:34.781890 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:34.781891 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:34.781950 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:34.782229 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:34.782229 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:34.782230 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:34.782358 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:34.782407 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:34.782676 (+ 269us) server_negotiation.cc:299] Negotiation successful
0925 07:25:34.782741 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260925 07:25:34.783057 12378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:34.774874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.0.0.1:59394)
0925 07:25:34.775024 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:34.775052 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:34.775486 (+ 434us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:34.775712 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:34.775715 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:34.775932 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:34.776096 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:34.776100 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:34.776759 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:34.776760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:34.777604 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:34.777606 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:34.778001 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:34.778002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:34.778113 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:34.778515 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:34.778522 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:34.780001 (+ 1479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:34.781619 (+ 1618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:34.781624 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:34.781625 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:34.781819 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:34.782008 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:34.782009 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:34.782009 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:34.782150 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:34.782417 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:34.782422 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:34.782556 (+ 134us) client_negotiation.cc:769] Sending connection context
0925 07:25:34.782658 (+ 102us) client_negotiation.cc:240] Negotiation successful
0925 07:25:34.782808 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":90,"threads_started":1}
I20260925 07:25:34.783634 12376 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.158.125" port: 44439 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 07:25:34 dist-test-slave-1drq krb5kdc[11799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321134, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.158.125@KRBTEST.COM
Sep 25 07:25:34 dist-test-slave-1drq krb5kdc[11799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321134, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.158.125@KRBTEST.COM for oryx/127.5.158.125@KRBTEST.COM
I20260925 07:25:34.788249 12380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:34.779844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42684 (local address 127.5.158.125:44439)
0925 07:25:34.779977 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:34.779982 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:34.780004 (+ 22us) server_negotiation.cc:407] Connection header received
0925 07:25:34.780055 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:34.780057 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:34.780071 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:34.780133 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:34.780828 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:34.781823 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:34.782791 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:34.783139 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:34.783206 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:34.785497 (+ 2291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:34.785504 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:34.785505 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:34.785511 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:34.787069 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:34.787447 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:34.787450 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:34.787450 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:34.787507 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:34.787706 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:34.787707 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:34.787707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:34.787849 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:34.787975 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:34.788066 (+ 91us) server_negotiation.cc:299] Negotiation successful
0925 07:25:34.788137 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":62,"threads_started":1}
I20260925 07:25:34.790700 12378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:34.783859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.125:44439 (local address 127.0.0.1:42698)
0925 07:25:34.784535 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:34.784544 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:34.784594 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:34.784713 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:34.784714 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:34.784803 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:34.784845 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:34.784845 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:34.785528 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:34.785528 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:34.785975 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:34.785977 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:34.786467 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:34.786468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:34.786540 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:34.786867 (+ 327us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:34.786871 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:34.788018 (+ 1147us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:34.789815 (+ 1797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:34.789816 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:34.789816 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:34.789964 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:34.790156 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:34.790157 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:34.790157 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:34.790206 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:34.790419 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:34.790420 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:34.790452 (+ 32us) client_negotiation.cc:769] Sending connection context
0925 07:25:34.790518 (+ 66us) client_negotiation.cc:240] Negotiation successful
0925 07:25:34.790600 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":662}
I20260925 07:25:34.790793 12381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:34.783927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42698 (local address 127.5.158.125:44439)
0925 07:25:34.784044 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:34.784046 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:34.784601 (+ 555us) server_negotiation.cc:407] Connection header received
0925 07:25:34.784625 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:34.784626 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:34.784635 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:34.784712 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:34.784903 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:34.785432 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:34.786038 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:34.786391 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:34.786457 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:34.788108 (+ 1651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:34.788109 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:34.788109 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:34.788115 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:34.789726 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:34.790037 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:34.790038 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:34.790038 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:34.790094 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:34.790272 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:34.790273 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:34.790273 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:34.790364 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:34.790426 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:34.790537 (+ 111us) server_negotiation.cc:299] Negotiation successful
0925 07:25:34.790647 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":101,"thread_start_us":80,"threads_started":1}
I20260925 07:25:34.791805 12376 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564: Bootstrap starting.
I20260925 07:25:34.792346 12376 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564: Neither blocks nor log segments found. Creating new log.
I20260925 07:25:34.792611 12376 log.cc:824] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564: Log is configured to *not* fsync() on all Append() calls
I20260925 07:25:34.793273 12376 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564: No bootstrap required, opened a new log
I20260925 07:25:34.794538 12376 raft_consensus.cc:348] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea66681bd57f42b0a9db2845ec74d84d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37781 } } peers { permanent_uuid: "87d6c9fb88664a3aa4245a34c108f564" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 44439 } }
I20260925 07:25:34.794649 12376 raft_consensus.cc:374] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:25:34.794696 12376 raft_consensus.cc:729] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87d6c9fb88664a3aa4245a34c108f564, State: Initialized, Role: FOLLOWER
I20260925 07:25:34.794803 12376 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea66681bd57f42b0a9db2845ec74d84d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37781 } } peers { permanent_uuid: "87d6c9fb88664a3aa4245a34c108f564" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 44439 } }
I20260925 07:25:34.794999 12383 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea66681bd57f42b0a9db2845ec74d84d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37781 } } peers { permanent_uuid: "87d6c9fb88664a3aa4245a34c108f564" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 44439 } } }
I20260925 07:25:34.795054 12376 sys_catalog.cc:564] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:25:34.795071 12383 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [sys.catalog]: This master's current role is: FOLLOWER
W20260925 07:25:34.796623 12394 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 07:25:34.796675 12394 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 07:25:34.798116 12308 master_runner.cc:428] Detected that this master 87d6c9fb88664a3aa4245a34c108f564 is joining an existing cluster
I20260925 07:25:34.798166 12308 master_runner.cc:432] Initiating AddMaster RPC to add 127.5.158.125:44439
I20260925 07:25:34.803915 11903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:34.799100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59404 (local address 127.5.158.126:37781)
0925 07:25:34.799141 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:34.799143 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:34.799153 (+ 10us) server_negotiation.cc:407] Connection header received
0925 07:25:34.799178 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:34.799179 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:34.799187 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:34.799234 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:34.799510 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:34.799960 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:34.800605 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:34.800796 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:34.800858 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:34.801640 (+ 782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:34.801641 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:34.801642 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:34.801647 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:34.802920 (+ 1273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:34.803249 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:34.803250 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:34.803250 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:34.803299 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:34.803491 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:34.803491 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:34.803492 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:34.803584 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:34.803640 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:34.803773 (+ 133us) server_negotiation.cc:299] Negotiation successful
0925 07:25:34.803832 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 07:25:34.803936 12400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:34.798929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.0.0.1:59404)
0925 07:25:34.799062 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:34.799072 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:34.799134 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:34.799237 (+ 103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:34.799238 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:34.799376 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:34.799442 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:34.799444 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:34.800029 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:34.800029 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:34.800534 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:34.800535 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:34.800874 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:34.800875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:34.800934 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:34.801302 (+ 368us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:34.801306 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:34.801548 (+ 242us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:34.803012 (+ 1464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:34.803013 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:34.803013 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:34.803167 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:34.803373 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:34.803374 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:34.803374 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:34.803423 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:34.803644 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:34.803644 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:34.803679 (+ 35us) client_negotiation.cc:769] Sending connection context
0925 07:25:34.803749 (+ 70us) client_negotiation.cc:240] Negotiation successful
0925 07:25:34.803833 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":78,"threads_started":1}
I20260925 07:25:34.809361 12381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:34.799062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42700 (local address 127.5.158.125:44439)
0925 07:25:34.799094 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:34.799095 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:34.804050 (+ 4955us) server_negotiation.cc:407] Connection header received
0925 07:25:34.804133 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:34.804134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:34.804142 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:34.804166 (+ 24us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:34.804449 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:34.804973 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:34.805845 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:34.806135 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:34.806196 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:34.807028 (+ 832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:34.807029 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:34.807029 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:34.807035 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:34.808468 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:34.808730 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:34.808731 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:34.808732 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:34.808774 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:34.808934 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:34.808935 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:34.808935 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:34.809036 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:34.809114 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:34.809217 (+ 103us) server_negotiation.cc:299] Negotiation successful
0925 07:25:34.809290 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 07:25:34.809474 12400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:34.798956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.125:44439 (local address 127.0.0.1:42700)
0925 07:25:34.803999 (+ 5043us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:34.804009 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:34.804071 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:34.804202 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:34.804203 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:34.804331 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:34.804376 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:34.804378 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:34.805193 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:34.805194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:34.805770 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:34.805772 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:34.806212 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:34.806213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:34.806267 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:34.806625 (+ 358us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:34.806629 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:34.806927 (+ 298us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:34.808527 (+ 1600us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:34.808527 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:34.808528 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:34.808668 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:34.808832 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:34.808833 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:34.808834 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:34.808879 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:34.809104 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:34.809105 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:34.809137 (+ 32us) client_negotiation.cc:769] Sending connection context
0925 07:25:34.809199 (+ 62us) client_negotiation.cc:240] Negotiation successful
0925 07:25:34.809356 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5033,"mutex_wait_us":64,"thread_start_us":59,"threads_started":1}
Sep 25 07:25:34 dist-test-slave-1drq krb5kdc[11799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321134, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.158.126@KRBTEST.COM for oryx/127.5.158.125@KRBTEST.COM
I20260925 07:25:34.813474 12380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:34.805914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42714 (local address 127.5.158.125:44439)
0925 07:25:34.805943 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:34.805944 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:34.806386 (+ 442us) server_negotiation.cc:407] Connection header received
0925 07:25:34.806707 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:34.806708 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:34.806727 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:34.806792 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:34.807534 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:34.808232 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:34.809270 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:34.809569 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:34.809633 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:34.811346 (+ 1713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:34.811348 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:34.811348 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:34.811354 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:34.812448 (+ 1094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:34.812779 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:34.812779 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:34.812780 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:34.812833 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:34.813005 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:34.813006 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:34.813006 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:34.813136 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:34.813189 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:34.813338 (+ 149us) server_negotiation.cc:299] Negotiation successful
0925 07:25:34.813404 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 07:25:34.813498 12402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:34.805879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.125:44439 (local address 127.0.0.1:42714)
0925 07:25:34.806264 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:34.806300 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:34.806620 (+ 320us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:34.806801 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:34.806803 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:34.807137 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:34.807444 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:34.807447 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:34.808323 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:34.808324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:34.809178 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:34.809181 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:34.809676 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:34.809678 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:34.809784 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:34.810144 (+ 360us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:34.810147 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:34.811272 (+ 1125us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:34.812520 (+ 1248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:34.812525 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:34.812528 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:34.812700 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:34.812893 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:34.812893 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:34.812894 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:34.812945 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:34.813184 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:34.813190 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:34.813233 (+ 43us) client_negotiation.cc:769] Sending connection context
0925 07:25:34.813316 (+ 83us) client_negotiation.cc:240] Negotiation successful
0925 07:25:34.813410 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":49,"threads_started":1}
I20260925 07:25:34.813890 11844 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.5.158.125:44439: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "87d6c9fb88664a3aa4245a34c108f564"
member_type: NON_VOTER
last_known_addr {
host: "127.5.158.125"
port: 44439
}
attrs {
promote: true
}
}
dest_uuid: "ea66681bd57f42b0a9db2845ec74d84d"
cas_config_opid_index: -1
I20260925 07:25:34.814152 11844 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea66681bd57f42b0a9db2845ec74d84d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ea66681bd57f42b0a9db2845ec74d84d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37781 } } peers { permanent_uuid: "87d6c9fb88664a3aa4245a34c108f564" member_type: NON_VOTER last_known_addr { host: "127.5.158.125" port: 44439 } attrs { promote: true } }
I20260925 07:25:34.814531 12404 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ea66681bd57f42b0a9db2845ec74d84d [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 87d6c9fb88664a3aa4245a34c108f564 (127.5.158.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ea66681bd57f42b0a9db2845ec74d84d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37781 } } peers { permanent_uuid: "87d6c9fb88664a3aa4245a34c108f564" member_type: NON_VOTER last_known_addr { host: "127.5.158.125" port: 44439 } attrs { promote: true } } }
I20260925 07:25:34.814723 12350 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:25:34.815366 12350 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [term 1 FOLLOWER]: Refusing update from remote peer ea66681bd57f42b0a9db2845ec74d84d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260925 07:25:34.815420 12404 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.5.158.125:44439
I20260925 07:25:34.815618 12308 master.cc:537] Master@127.5.158.125:44439 shutting down...
I20260925 07:25:34.815763 12403 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ea66681bd57f42b0a9db2845ec74d84d [LEADER]: Connected to new peer: Peer: permanent_uuid: "87d6c9fb88664a3aa4245a34c108f564" member_type: NON_VOTER last_known_addr { host: "127.5.158.125" port: 44439 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:25:34.816157 12405 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea66681bd57f42b0a9db2845ec74d84d [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ea66681bd57f42b0a9db2845ec74d84d" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ea66681bd57f42b0a9db2845ec74d84d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37781 } } peers { permanent_uuid: "87d6c9fb88664a3aa4245a34c108f564" member_type: NON_VOTER last_known_addr { host: "127.5.158.125" port: 44439 } attrs { promote: true } } }
I20260925 07:25:34.816229 12405 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea66681bd57f42b0a9db2845ec74d84d [sys.catalog]: This master's current role is: LEADER
W20260925 07:25:34.816366 11830 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260925 07:25:34.816749 11830 consensus_peers.cc:487] T 00000000000000000000000000000000 P ea66681bd57f42b0a9db2845ec74d84d -> Peer 87d6c9fb88664a3aa4245a34c108f564 (127.5.158.125:44439): Couldn't send request to peer 87d6c9fb88664a3aa4245a34c108f564. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260925 07:25:34.818284 12308 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 07:25:34.818368 12308 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:25:34.818408 12308 tablet_replica.cc:330] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564: stopping tablet replica
I20260925 07:25:34.830762 12308 master.cc:552] Master@127.5.158.125:44439 shutdown complete.
I20260925 07:25:34.830803 12308 master_runner.cc:305] Clearing existing system tablet
I20260925 07:25:34.830931 12308 fs_manager.cc:359] Metadata directory not provided
I20260925 07:25:34.830973 12308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:25:34.831557 12308 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.002s
I20260925 07:25:34.832013 12409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:34.832172 12308 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:25:34.832237 12308 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:34.832267 12308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-1/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-1/wal
uuid: "87d6c9fb88664a3aa4245a34c108f564"
format_stamp: "Formatted at 2026-09-25 07:25:34 on dist-test-slave-1drq"
server_key: "4f406b533db55a9d62f5a97656932132"
server_key_iv: "fb4bc136b3a5518fe6be0fd6826bd2ed"
server_key_version: "encryptionkey@0"
I20260925 07:25:34.832729 12308 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:25:34.834014 12308 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260925 07:25:34.834062 12308 log.cc:1196] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260925 07:25:34.834229 12308 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564: Deleting consensus metadata
I20260925 07:25:34.834342 12308 master_runner.cc:315] Copying system tablet from 127.5.158.126:37781
I20260925 07:25:34.834847 12308 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564: tablet copy: Beginning tablet copy session from remote peer at address 127.5.158.126:37781
I20260925 07:25:34.840072 11903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:34.835100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59408 (local address 127.5.158.126:37781)
0925 07:25:34.835130 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:34.835131 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:34.835254 (+ 123us) server_negotiation.cc:407] Connection header received
0925 07:25:34.835338 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:34.835339 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:34.835347 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:34.835398 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:34.835649 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:34.836095 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:34.836726 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:34.836901 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:34.836960 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:34.837802 (+ 842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:34.837803 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:34.837803 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:34.837810 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:34.839000 (+ 1190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:34.839409 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:34.839411 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:34.839411 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:34.839460 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:34.839650 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:34.839651 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:34.839652 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:34.839747 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:34.839806 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:34.839937 (+ 131us) server_negotiation.cc:299] Negotiation successful
0925 07:25:34.839998 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
I20260925 07:25:34.840091 12415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:34.835081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.0.0.1:59408)
0925 07:25:34.835181 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:34.835189 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:34.835268 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:34.835404 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:34.835404 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:34.835539 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:34.835581 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:34.835582 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:34.836166 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:34.836167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:34.836653 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:34.836654 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:34.836971 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:34.836972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:34.837072 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:34.837484 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:34.837488 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:34.837722 (+ 234us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:34.839092 (+ 1370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:34.839093 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:34.839093 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:34.839306 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:34.839526 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:34.839527 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:34.839527 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:34.839579 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:34.839810 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:34.839810 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:34.839846 (+ 36us) client_negotiation.cc:769] Sending connection context
0925 07:25:34.839916 (+ 70us) client_negotiation.cc:240] Negotiation successful
0925 07:25:34.839999 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76,"thread_start_us":61,"threads_started":1}
I20260925 07:25:34.840430 11864 tablet_copy_service.cc:140] P ea66681bd57f42b0a9db2845ec74d84d: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 87d6c9fb88664a3aa4245a34c108f564 ({username='oryx', principal='oryx/127.5.158.125@KRBTEST.COM'} at 127.0.0.1:59408)
I20260925 07:25:34.840508 11864 tablet_copy_service.cc:161] P ea66681bd57f42b0a9db2845ec74d84d: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 87d6c9fb88664a3aa4245a34c108f564 at {username='oryx', principal='oryx/127.5.158.125@KRBTEST.COM'} at 127.0.0.1:59408: session id = 87d6c9fb88664a3aa4245a34c108f564-00000000000000000000000000000000
I20260925 07:25:34.841197 11864 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P ea66681bd57f42b0a9db2845ec74d84d: Tablet Copy: opened 0 blocks and 1 log segments
I20260925 07:25:34.841734 12308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260925 07:25:34.842478 12308 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564: tablet copy: Starting download of 0 data blocks...
I20260925 07:25:34.842593 12308 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564: tablet copy: Starting download of 1 WAL segments...
I20260925 07:25:34.843261 12308 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260925 07:25:34.843896 11864 tablet_copy_service.cc:342] P ea66681bd57f42b0a9db2845ec74d84d: Request end of tablet copy session 87d6c9fb88664a3aa4245a34c108f564-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.5.158.125@KRBTEST.COM'} at 127.0.0.1:59408
I20260925 07:25:34.843936 11864 tablet_copy_service.cc:433] P ea66681bd57f42b0a9db2845ec74d84d: ending tablet copy session 87d6c9fb88664a3aa4245a34c108f564-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 87d6c9fb88664a3aa4245a34c108f564
I20260925 07:25:34.845753 12308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:25:34.846719 12416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:34.846752 12417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:34.846807 12419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:34.847195 12308 server_base.cc:1034] running on GCE node
I20260925 07:25:34.847259 12308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:25:34.847383 12308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:25:34.848510 12308 hybrid_clock.cc:648] HybridClock initialized: now 1790321134848502 us; error 23 us; skew 500 ppm
Sep 25 07:25:34 dist-test-slave-1drq krb5kdc[11799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321134, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.158.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:25:34.849529 12308 init.cc:377] Logged in from keytab as oryx/127.5.158.125@KRBTEST.COM (short username oryx)
I20260925 07:25:34.849767 12308 webserver.cc:460] Webserver started at http://127.5.158.125:43475/ using document root <none> and password file <none>
I20260925 07:25:34.849867 12308 fs_manager.cc:359] Metadata directory not provided
I20260925 07:25:34.849897 12308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:25:34.850386 12308 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:25:34.850812 12426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:34.850919 12308 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:34.850960 12308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-1/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-1/wal
uuid: "87d6c9fb88664a3aa4245a34c108f564"
format_stamp: "Formatted at 2026-09-25 07:25:34 on dist-test-slave-1drq"
server_key: "4f406b533db55a9d62f5a97656932132"
server_key_iv: "fb4bc136b3a5518fe6be0fd6826bd2ed"
server_key_version: "encryptionkey@0"
I20260925 07:25:34.851001 12308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:25:34.864513 12308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:25:34.865854 12308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:25:34.868782 12308 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.125:44439
I20260925 07:25:34.869078 12478 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.125:44439 every 8 connection(s)
I20260925 07:25:34.869148 12308 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-1/data/info.pb
I20260925 07:25:34.869724 12479 sys_catalog.cc:263] Verifying existing consensus state
I20260925 07:25:34.870088 12479 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564: Bootstrap starting.
I20260925 07:25:34.872552 12479 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 07:25:34.872784 12479 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564: Bootstrap complete.
I20260925 07:25:34.872968 12479 raft_consensus.cc:348] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ea66681bd57f42b0a9db2845ec74d84d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37781 } } peers { permanent_uuid: "87d6c9fb88664a3aa4245a34c108f564" member_type: NON_VOTER last_known_addr { host: "127.5.158.125" port: 44439 } attrs { promote: true } }
I20260925 07:25:34.873059 12479 raft_consensus.cc:729] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 87d6c9fb88664a3aa4245a34c108f564, State: Initialized, Role: LEARNER
I20260925 07:25:34.873121 12479 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ea66681bd57f42b0a9db2845ec74d84d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37781 } } peers { permanent_uuid: "87d6c9fb88664a3aa4245a34c108f564" member_type: NON_VOTER last_known_addr { host: "127.5.158.125" port: 44439 } attrs { promote: true } }
I20260925 07:25:34.873309 12479 sys_catalog.cc:564] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:25:34.873315 12482 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ea66681bd57f42b0a9db2845ec74d84d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37781 } } peers { permanent_uuid: "87d6c9fb88664a3aa4245a34c108f564" member_type: NON_VOTER last_known_addr { host: "127.5.158.125" port: 44439 } attrs { promote: true } } }
I20260925 07:25:34.873435 12482 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [sys.catalog]: This master's current role is: LEARNER
I20260925 07:25:34.874819 12493 catalog_manager.cc:1259] Loaded cluster ID: c10bd3f7239e4d6aaf544a03c0ac2c01
I20260925 07:25:34.874864 12493 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564: loading cluster ID for follower catalog manager: success
I20260925 07:25:34.875761 12493 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564: acquiring CA information for follower catalog manager: success
I20260925 07:25:34.876013 12493 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 07:25:34.895450 12494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:34.889695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42738 (local address 127.5.158.125:44439)
0925 07:25:34.889810 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:34.889812 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:34.889820 (+ 8us) server_negotiation.cc:407] Connection header received
0925 07:25:34.889843 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:34.889844 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:34.889851 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:34.889918 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:34.890389 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:34.891003 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:34.891691 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:34.891878 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:34.891943 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:34.893015 (+ 1072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:34.893016 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:34.893016 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:34.893022 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:34.894352 (+ 1330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:34.894683 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:34.894684 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:34.894685 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:34.894741 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:34.894929 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:34.894929 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:34.894930 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:34.895026 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:34.895114 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:34.895223 (+ 109us) server_negotiation.cc:299] Negotiation successful
0925 07:25:34.895333 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":55,"threads_started":1}
WARNING: no policy specified for oryx/127.5.158.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.158.124@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.158.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/oryx_127.5.158.124.keytab.
Entry for principal oryx/127.5.158.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/oryx_127.5.158.124.keytab.
I20260925 07:25:34.917478 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.124:36829
--webserver_interface=127.5.158.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/oryx_127.5.158.124.keytab
--principal=oryx/127.5.158.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:45003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.158.126:37781,127.5.158.125:44439,127.5.158.124:36829
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:25:34.992087 12497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:25:34.992278 12497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:25:34.992313 12497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:25:34.993403 12497 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:25:34.993459 12497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:25:34.993484 12497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:25:34.993513 12497 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:25:34.993535 12497 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:25:34.995035 12497 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:45003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.158.126:37781,127.5.158.125:44439,127.5.158.124:36829
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/oryx_127.5.158.124.keytab
--principal=oryx/127.5.158.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.124:36829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.158.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12497
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:25:34.995283 12497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:25:34.995491 12497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:25:35.000100 12503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:35.000066 12505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:35.000078 12502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:35.000236 12497 server_base.cc:1034] running on GCE node
I20260925 07:25:35.000469 12497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:25:35.000721 12497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:25:35.001871 12497 hybrid_clock.cc:648] HybridClock initialized: now 1790321135001863 us; error 24 us; skew 500 ppm
Sep 25 07:25:35 dist-test-slave-1drq krb5kdc[11799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321135, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.158.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:25:35.003906 12497 init.cc:377] Logged in from keytab as oryx/127.5.158.124@KRBTEST.COM (short username oryx)
I20260925 07:25:35.004287 12497 webserver.cc:460] Webserver started at http://127.5.158.124:46245/ using document root <none> and password file <none>
I20260925 07:25:35.004447 12497 fs_manager.cc:359] Metadata directory not provided
I20260925 07:25:35.004482 12497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:25:35.004552 12497 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:25:35.005412 12497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-2/data/instance:
uuid: "52179093193647ce9b274a063c18597b"
format_stamp: "Formatted at 2026-09-25 07:25:35 on dist-test-slave-1drq"
server_key: "6f80e6d3f98ccd609a831f67d6db0355"
server_key_iv: "a51da98c8728dd99d941f68f1a427c37"
server_key_version: "encryptionkey@0"
I20260925 07:25:35.005671 12497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-2/wal/instance:
uuid: "52179093193647ce9b274a063c18597b"
format_stamp: "Formatted at 2026-09-25 07:25:35 on dist-test-slave-1drq"
server_key: "6f80e6d3f98ccd609a831f67d6db0355"
server_key_iv: "a51da98c8728dd99d941f68f1a427c37"
server_key_version: "encryptionkey@0"
I20260925 07:25:35.006724 12497 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 07:25:35.007303 12512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:35.007445 12497 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:25:35.007524 12497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-2/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-2/wal
uuid: "52179093193647ce9b274a063c18597b"
format_stamp: "Formatted at 2026-09-25 07:25:35 on dist-test-slave-1drq"
server_key: "6f80e6d3f98ccd609a831f67d6db0355"
server_key_iv: "a51da98c8728dd99d941f68f1a427c37"
server_key_version: "encryptionkey@0"
I20260925 07:25:35.007576 12497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:25:35.029346 12497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:25:35.031280 12497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:25:35.031373 12497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:25:35.034921 12497 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.124:36829
I20260925 07:25:35.034978 12564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.124:36829 every 8 connection(s)
I20260925 07:25:35.035260 12497 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-2/data/info.pb
I20260925 07:25:35.036051 12565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:25:35.037397 12565 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37781 } has no permanent_uuid. Determining permanent_uuid...
I20260925 07:25:35.041303 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 12497
Sep 25 07:25:35 dist-test-slave-1drq krb5kdc[11799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321135, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.158.124@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM
I20260925 07:25:35.045150 11903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:35.038073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59418 (local address 127.5.158.126:37781)
0925 07:25:35.038145 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:35.038147 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:35.038240 (+ 93us) server_negotiation.cc:407] Connection header received
0925 07:25:35.038619 (+ 379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:35.038620 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:35.038629 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:35.038667 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:35.039139 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.039594 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.040423 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.040601 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.040647 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:35.042587 (+ 1940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:35.042589 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:35.042589 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:35.042596 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:35.043803 (+ 1207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:35.044162 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:35.044164 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:35.044164 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:35.044219 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:35.044457 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:35.044498 (+ 41us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:35.044498 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:35.044615 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:35.044689 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:35.044935 (+ 246us) server_negotiation.cc:299] Negotiation successful
0925 07:25:35.045028 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 07:25:35.045265 12567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:35.037947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.0.0.1:59418)
0925 07:25:35.038117 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:35.038163 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:35.038542 (+ 379us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:35.038727 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:35.038730 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:35.038922 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:35.039081 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:35.039084 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.039660 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.039660 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:35.040358 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:35.040360 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.040677 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.040677 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:35.040777 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:35.041154 (+ 377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:35.041161 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:35.042463 (+ 1302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:35.043893 (+ 1430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:35.043899 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:35.043901 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:35.044093 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:35.044278 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:35.044279 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:35.044279 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:35.044375 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:35.044680 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:35.044686 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:35.044813 (+ 127us) client_negotiation.cc:769] Sending connection context
0925 07:25:35.044910 (+ 97us) client_negotiation.cc:240] Negotiation successful
0925 07:25:35.045042 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":78,"threads_started":1}
I20260925 07:25:35.045778 12565 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.158.125" port: 44439 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 07:25:35 dist-test-slave-1drq krb5kdc[11799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321134, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.158.124@KRBTEST.COM
I20260925 07:25:35.049113 12568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:35.041780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38410 (local address 127.5.158.124:36829)
0925 07:25:35.041914 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:35.041918 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:35.041935 (+ 17us) server_negotiation.cc:407] Connection header received
0925 07:25:35.041952 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:35.041954 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:35.041973 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:35.042038 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:35.042572 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.043146 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.043767 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.044030 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.044096 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:35.046607 (+ 2511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:35.046612 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:35.046612 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:35.046618 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:35.048062 (+ 1444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:35.048385 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:35.048388 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:35.048388 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:35.048438 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:35.048612 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:35.048613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:35.048613 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:35.048732 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:35.048859 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:35.048933 (+ 74us) server_negotiation.cc:299] Negotiation successful
0925 07:25:35.048997 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":68,"threads_started":1}
Sep 25 07:25:35 dist-test-slave-1drq krb5kdc[11799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321135, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.158.124@KRBTEST.COM for oryx/127.5.158.125@KRBTEST.COM
I20260925 07:25:35.051591 12567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:35.045979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.125:44439 (local address 127.0.0.1:42754)
0925 07:25:35.046023 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:35.046033 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:35.046066 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:35.046207 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:35.046208 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:35.046340 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:35.046381 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:35.046382 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.047266 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.047267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:35.047855 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:35.047857 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.048310 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.048311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:35.048376 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:35.048706 (+ 330us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:35.048710 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:35.049734 (+ 1024us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:35.050902 (+ 1168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:35.050903 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:35.050903 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:35.051036 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:35.051205 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:35.051206 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:35.051206 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:35.051260 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:35.051420 (+ 160us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:35.051420 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:35.051447 (+ 27us) client_negotiation.cc:769] Sending connection context
0925 07:25:35.051492 (+ 45us) client_negotiation.cc:240] Negotiation successful
0925 07:25:35.051545 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260925 07:25:35.051638 12494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:35.046046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42754 (local address 127.5.158.125:44439)
0925 07:25:35.046094 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:35.046096 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:35.046106 (+ 10us) server_negotiation.cc:407] Connection header received
0925 07:25:35.046132 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:35.046133 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:35.046143 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:35.046186 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:35.046474 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.047173 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.047935 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.048247 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.048312 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:35.049796 (+ 1484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:35.049797 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:35.049797 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:35.049803 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:35.050849 (+ 1046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:35.051082 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:35.051083 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:35.051083 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:35.051143 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:35.051297 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:35.051297 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:35.051297 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:35.051377 (+ 80us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:35.051418 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:35.051504 (+ 86us) server_negotiation.cc:299] Negotiation successful
0925 07:25:35.051568 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 07:25:35.052031 12565 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.158.124" port: 36829 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 07:25:35 dist-test-slave-1drq krb5kdc[11799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321135, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.158.124@KRBTEST.COM for oryx/127.5.158.124@KRBTEST.COM
I20260925 07:25:35.057310 12567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:35.052206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.124:36829 (local address 127.0.0.1:38424)
0925 07:25:35.052259 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:35.052268 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:35.052310 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:35.052414 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:35.052415 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:35.052523 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:35.052541 (+ 18us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:35.052542 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.053064 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.053065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:35.053489 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:35.053490 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.053809 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.053810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:35.053851 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:35.054173 (+ 322us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:35.054184 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:35.055078 (+ 894us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:35.056397 (+ 1319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:35.056397 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:35.056397 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:35.056535 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:35.056716 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:35.056716 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:35.056717 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:35.056762 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:35.056959 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:35.056959 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:35.056990 (+ 31us) client_negotiation.cc:769] Sending connection context
0925 07:25:35.057084 (+ 94us) client_negotiation.cc:240] Negotiation successful
0925 07:25:35.057192 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260925 07:25:35.057363 12568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:35.052241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38424 (local address 127.5.158.124:36829)
0925 07:25:35.052270 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:35.052272 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:35.052298 (+ 26us) server_negotiation.cc:407] Connection header received
0925 07:25:35.052357 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:35.052357 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:35.052365 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:35.052411 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:35.052606 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.052992 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.053566 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.053740 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.053809 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:35.055147 (+ 1338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:35.055148 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:35.055149 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:35.055154 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:35.056313 (+ 1159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:35.056603 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:35.056604 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:35.056604 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:35.056648 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:35.056819 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:35.056820 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:35.056821 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:35.056906 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:35.056965 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:35.057125 (+ 160us) server_negotiation.cc:299] Negotiation successful
0925 07:25:35.057240 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":16}
I20260925 07:25:35.058331 12565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52179093193647ce9b274a063c18597b: Bootstrap starting.
I20260925 07:25:35.059087 12565 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 52179093193647ce9b274a063c18597b: Neither blocks nor log segments found. Creating new log.
I20260925 07:25:35.059335 12565 log.cc:824] T 00000000000000000000000000000000 P 52179093193647ce9b274a063c18597b: Log is configured to *not* fsync() on all Append() calls
I20260925 07:25:35.059894 12565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52179093193647ce9b274a063c18597b: No bootstrap required, opened a new log
I20260925 07:25:35.061197 12565 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52179093193647ce9b274a063c18597b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea66681bd57f42b0a9db2845ec74d84d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37781 } } peers { permanent_uuid: "87d6c9fb88664a3aa4245a34c108f564" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 44439 } } peers { permanent_uuid: "52179093193647ce9b274a063c18597b" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 36829 } }
I20260925 07:25:35.061290 12565 raft_consensus.cc:374] T 00000000000000000000000000000000 P 52179093193647ce9b274a063c18597b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:25:35.061324 12565 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52179093193647ce9b274a063c18597b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52179093193647ce9b274a063c18597b, State: Initialized, Role: FOLLOWER
I20260925 07:25:35.061432 12565 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52179093193647ce9b274a063c18597b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea66681bd57f42b0a9db2845ec74d84d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37781 } } peers { permanent_uuid: "87d6c9fb88664a3aa4245a34c108f564" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 44439 } } peers { permanent_uuid: "52179093193647ce9b274a063c18597b" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 36829 } }
I20260925 07:25:35.061673 12565 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52179093193647ce9b274a063c18597b [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:25:35.061646 12570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52179093193647ce9b274a063c18597b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea66681bd57f42b0a9db2845ec74d84d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37781 } } peers { permanent_uuid: "87d6c9fb88664a3aa4245a34c108f564" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 44439 } } peers { permanent_uuid: "52179093193647ce9b274a063c18597b" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 36829 } } }
I20260925 07:25:35.061736 12570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52179093193647ce9b274a063c18597b [sys.catalog]: This master's current role is: FOLLOWER
W20260925 07:25:35.063022 12581 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 52179093193647ce9b274a063c18597b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 07:25:35.063069 12581 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 07:25:35.064965 12497 master_runner.cc:428] Detected that this master 52179093193647ce9b274a063c18597b is joining an existing cluster
I20260925 07:25:35.065001 12497 master_runner.cc:432] Initiating AddMaster RPC to add 127.5.158.124:36829
I20260925 07:25:35.072117 11903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:35.065901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59426 (local address 127.5.158.126:37781)
0925 07:25:35.065958 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:35.065960 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:35.066189 (+ 229us) server_negotiation.cc:407] Connection header received
0925 07:25:35.066277 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:35.066278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:35.066288 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:35.066336 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:35.066753 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.067492 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.068664 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.068923 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.068960 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:35.069895 (+ 935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:35.069896 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:35.069896 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:35.069901 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:35.071159 (+ 1258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:35.071479 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:35.071480 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:35.071480 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:35.071534 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:35.071733 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:35.071733 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:35.071734 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:35.071823 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:35.071865 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:35.072019 (+ 154us) server_negotiation.cc:299] Negotiation successful
0925 07:25:35.072067 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260925 07:25:35.072428 12588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:35.065962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.0.0.1:59426)
0925 07:25:35.066108 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:35.066118 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:35.066218 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:35.066343 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:35.066344 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:35.066611 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:35.066670 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:35.066671 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.067586 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.067587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:35.068574 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:35.068576 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.069022 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.069023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:35.069090 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:35.069435 (+ 345us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:35.069439 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:35.069813 (+ 374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:35.071224 (+ 1411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:35.071225 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:35.071225 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:35.071416 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:35.071588 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:35.071589 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:35.071589 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:35.071645 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:35.071873 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:35.071874 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:35.071921 (+ 47us) client_negotiation.cc:769] Sending connection context
0925 07:25:35.072001 (+ 80us) client_negotiation.cc:240] Negotiation successful
0925 07:25:35.072083 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":92,"thread_start_us":71,"threads_started":1}
Sep 25 07:25:35 dist-test-slave-1drq krb5kdc[11799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321134, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.158.126@KRBTEST.COM for oryx/127.5.158.124@KRBTEST.COM
I20260925 07:25:35.088351 12402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:35.074088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.124:36829 (local address 127.0.0.1:38444)
0925 07:25:35.074129 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:35.074139 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:35.074179 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:35.074445 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:35.074446 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:35.074590 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:35.074629 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:35.074630 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.075390 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.075391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:35.076145 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:35.076147 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.076473 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.076474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:35.076531 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:35.076817 (+ 286us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:35.076822 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:35.078037 (+ 1215us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:35.087275 (+ 9238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:35.087276 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:35.087276 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:35.087495 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:35.087705 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:35.087706 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:35.087706 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:35.087764 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:35.088048 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:35.088049 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:35.088093 (+ 44us) client_negotiation.cc:769] Sending connection context
0925 07:25:35.088139 (+ 46us) client_negotiation.cc:240] Negotiation successful
0925 07:25:35.088249 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260925 07:25:35.089052 12589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:35.066089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.124:36829 (local address 127.0.0.1:38438)
0925 07:25:35.066421 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:35.066447 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:35.066507 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:35.066665 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:35.066665 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:35.066831 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:35.066877 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:35.066878 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.067697 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.067698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:35.068549 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:35.068551 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.068993 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.068994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:35.069054 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:35.085448 (+ 16394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:35.085453 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:35.085807 (+ 354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:35.088055 (+ 2248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:35.088056 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:35.088056 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:35.088280 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:35.088436 (+ 156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:35.088437 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:35.088437 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:35.088477 (+ 40us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:35.088688 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:35.088689 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:35.088717 (+ 28us) client_negotiation.cc:769] Sending connection context
0925 07:25:35.088783 (+ 66us) client_negotiation.cc:240] Negotiation successful
0925 07:25:35.088882 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"mutex_wait_us":65,"thread_start_us":54,"threads_started":1}
I20260925 07:25:35.089265 12568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:35.066199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38438 (local address 127.5.158.124:36829)
0925 07:25:35.066225 (+ 26us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:35.066227 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:35.066489 (+ 262us) server_negotiation.cc:407] Connection header received
0925 07:25:35.066579 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:35.066579 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:35.066586 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:35.066609 (+ 23us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:35.066943 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.067626 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.068630 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.068887 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.068921 (+ 34us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:35.085885 (+ 16964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:35.085885 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:35.085886 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:35.085890 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:35.087976 (+ 2086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:35.088344 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:35.088345 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:35.088345 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:35.088384 (+ 39us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:35.088527 (+ 143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:35.088527 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:35.088527 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:35.088624 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:35.088670 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:35.088840 (+ 170us) server_negotiation.cc:299] Negotiation successful
0925 07:25:35.088934 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":14}
I20260925 07:25:35.089633 12587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:35.065893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.125:44439 (local address 127.0.0.1:42770)
0925 07:25:35.066055 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:35.066064 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:35.066133 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:35.066295 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:35.066296 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:35.066595 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:35.066702 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:35.066704 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.067614 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.067615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:35.068549 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:35.068551 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.068995 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.068995 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:35.069052 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:35.086168 (+ 17116us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:35.086172 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:35.086682 (+ 510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:35.088569 (+ 1887us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:35.088570 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:35.088571 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:35.088780 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:35.088969 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:35.088970 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:35.088970 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:35.089021 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:35.089321 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:35.089322 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:35.089365 (+ 43us) client_negotiation.cc:769] Sending connection context
0925 07:25:35.089432 (+ 67us) client_negotiation.cc:240] Negotiation successful
0925 07:25:35.089537 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":76,"thread_start_us":76,"threads_started":1}
I20260925 07:25:35.089651 12494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:35.065915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42770 (local address 127.5.158.125:44439)
0925 07:25:35.065988 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:35.065989 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:35.066110 (+ 121us) server_negotiation.cc:407] Connection header received
0925 07:25:35.066200 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:35.066201 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:35.066209 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:35.066238 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:35.066782 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.067531 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.068634 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.068887 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.068921 (+ 34us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:35.086776 (+ 17855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:35.086777 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:35.086778 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:35.086783 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:35.088486 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:35.088856 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:35.088857 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:35.088857 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:35.088905 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:35.089120 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:35.089121 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:35.089121 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:35.089244 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:35.089295 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:35.089473 (+ 178us) server_negotiation.cc:299] Negotiation successful
0925 07:25:35.089568 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260925 07:25:35.089900 12590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:35.074151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38444 (local address 127.5.158.124:36829)
0925 07:25:35.074299 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:35.074301 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:35.074330 (+ 29us) server_negotiation.cc:407] Connection header received
0925 07:25:35.074360 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:35.074360 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:35.074368 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:35.074426 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:35.074761 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.075324 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.076224 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.076401 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.076458 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:35.078136 (+ 1678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:35.078138 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:35.078138 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:35.078144 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:35.087162 (+ 9018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:35.087572 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:35.087573 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:35.087578 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:35.087642 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:35.087824 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:35.087825 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:35.087825 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:35.087948 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:35.088721 (+ 773us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:35.088757 (+ 36us) server_negotiation.cc:299] Negotiation successful
0925 07:25:35.088947 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":98,"threads_started":1}
I20260925 07:25:35.090471 11844 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.5.158.124:36829: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "52179093193647ce9b274a063c18597b"
member_type: NON_VOTER
last_known_addr {
host: "127.5.158.124"
port: 36829
}
attrs {
promote: true
}
}
dest_uuid: "ea66681bd57f42b0a9db2845ec74d84d"
cas_config_opid_index: 5
I20260925 07:25:35.090701 11844 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea66681bd57f42b0a9db2845ec74d84d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ea66681bd57f42b0a9db2845ec74d84d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37781 } } peers { permanent_uuid: "87d6c9fb88664a3aa4245a34c108f564" member_type: NON_VOTER last_known_addr { host: "127.5.158.125" port: 44439 } attrs { promote: true } } peers { permanent_uuid: "52179093193647ce9b274a063c18597b" member_type: NON_VOTER last_known_addr { host: "127.5.158.124" port: 36829 } attrs { promote: true } }
I20260925 07:25:35.090983 12405 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ea66681bd57f42b0a9db2845ec74d84d [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 52179093193647ce9b274a063c18597b (127.5.158.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ea66681bd57f42b0a9db2845ec74d84d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37781 } } peers { permanent_uuid: "87d6c9fb88664a3aa4245a34c108f564" member_type: NON_VOTER last_known_addr { host: "127.5.158.125" port: 44439 } attrs { promote: true } } peers { permanent_uuid: "52179093193647ce9b274a063c18597b" member_type: NON_VOTER last_known_addr { host: "127.5.158.124" port: 36829 } attrs { promote: true } } }
I20260925 07:25:35.091260 12539 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52179093193647ce9b274a063c18597b [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:25:35.091780 12405 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.5.158.124:36829
I20260925 07:25:35.091845 12404 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea66681bd57f42b0a9db2845ec74d84d [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ea66681bd57f42b0a9db2845ec74d84d" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ea66681bd57f42b0a9db2845ec74d84d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37781 } } peers { permanent_uuid: "87d6c9fb88664a3aa4245a34c108f564" member_type: NON_VOTER last_known_addr { host: "127.5.158.125" port: 44439 } attrs { promote: true } } peers { permanent_uuid: "52179093193647ce9b274a063c18597b" member_type: NON_VOTER last_known_addr { host: "127.5.158.124" port: 36829 } attrs { promote: true } } }
I20260925 07:25:35.091904 12404 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea66681bd57f42b0a9db2845ec74d84d [sys.catalog]: This master's current role is: LEADER
I20260925 07:25:35.091984 12539 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 52179093193647ce9b274a063c18597b [term 1 FOLLOWER]: Refusing update from remote peer ea66681bd57f42b0a9db2845ec74d84d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260925 07:25:35.092077 12497 master.cc:537] Master@127.5.158.124:36829 shutting down...
I20260925 07:25:35.092480 12404 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ea66681bd57f42b0a9db2845ec74d84d [LEADER]: Connected to new peer: Peer: permanent_uuid: "52179093193647ce9b274a063c18597b" member_type: NON_VOTER last_known_addr { host: "127.5.158.124" port: 36829 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 07:25:35.093209 11830 consensus_peers.cc:487] T 00000000000000000000000000000000 P ea66681bd57f42b0a9db2845ec74d84d -> Peer 52179093193647ce9b274a063c18597b (127.5.158.124:36829): Couldn't send request to peer 52179093193647ce9b274a063c18597b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260925 07:25:35.094617 12497 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 52179093193647ce9b274a063c18597b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 07:25:35.094708 12497 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 52179093193647ce9b274a063c18597b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:25:35.094753 12497 tablet_replica.cc:330] T 00000000000000000000000000000000 P 52179093193647ce9b274a063c18597b: stopping tablet replica
I20260925 07:25:35.096685 12494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:35.091118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42780 (local address 127.5.158.125:44439)
0925 07:25:35.091206 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:35.091208 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:35.093293 (+ 2085us) server_negotiation.cc:407] Connection header received
0925 07:25:35.093354 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:35.093355 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:35.093376 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:35.093432 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:25:35.093565 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.094249 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.095713 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.096278 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.096340 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:35.096393 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:35.096574 (+ 181us) server_negotiation.cc:299] Negotiation successful
0925 07:25:35.096605 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260925 07:25:35.096663 12402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:35.091067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.125:44439 (local address 127.0.0.1:42780)
0925 07:25:35.093241 (+ 2174us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:35.093251 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:35.093300 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:35.093449 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:35.093450 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:35.093451 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:25:35.093486 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:35.093488 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.094342 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.094343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:35.095635 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:35.095638 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.096424 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.096425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:35.096487 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:35.096506 (+ 19us) client_negotiation.cc:769] Sending connection context
0925 07:25:35.096561 (+ 55us) client_negotiation.cc:240] Negotiation successful
0925 07:25:35.096592 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2160}
I20260925 07:25:35.097024 12453 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [term 1 LEARNER]: Refusing update from remote peer ea66681bd57f42b0a9db2845ec74d84d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260925 07:25:35.097229 12404 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ea66681bd57f42b0a9db2845ec74d84d [LEADER]: Connected to new peer: Peer: permanent_uuid: "87d6c9fb88664a3aa4245a34c108f564" member_type: NON_VOTER last_known_addr { host: "127.5.158.125" port: 44439 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260925 07:25:35.097721 12453 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 52179093193647ce9b274a063c18597b (127.5.158.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ea66681bd57f42b0a9db2845ec74d84d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37781 } } peers { permanent_uuid: "87d6c9fb88664a3aa4245a34c108f564" member_type: NON_VOTER last_known_addr { host: "127.5.158.125" port: 44439 } attrs { promote: true } } peers { permanent_uuid: "52179093193647ce9b274a063c18597b" member_type: NON_VOTER last_known_addr { host: "127.5.158.124" port: 36829 } attrs { promote: true } } }
I20260925 07:25:35.098694 12592 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ea66681bd57f42b0a9db2845ec74d84d" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ea66681bd57f42b0a9db2845ec74d84d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37781 } } peers { permanent_uuid: "87d6c9fb88664a3aa4245a34c108f564" member_type: NON_VOTER last_known_addr { host: "127.5.158.125" port: 44439 } attrs { promote: true } } peers { permanent_uuid: "52179093193647ce9b274a063c18597b" member_type: NON_VOTER last_known_addr { host: "127.5.158.124" port: 36829 } attrs { promote: true } } }
I20260925 07:25:35.098789 12592 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [sys.catalog]: This master's current role is: LEARNER
I20260925 07:25:35.098874 12482 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea66681bd57f42b0a9db2845ec74d84d. Latest consensus state: current_term: 1 leader_uuid: "ea66681bd57f42b0a9db2845ec74d84d" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ea66681bd57f42b0a9db2845ec74d84d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37781 } } peers { permanent_uuid: "87d6c9fb88664a3aa4245a34c108f564" member_type: NON_VOTER last_known_addr { host: "127.5.158.125" port: 44439 } attrs { promote: true } } peers { permanent_uuid: "52179093193647ce9b274a063c18597b" member_type: NON_VOTER last_known_addr { host: "127.5.158.124" port: 36829 } attrs { promote: true } } }
I20260925 07:25:35.098919 12482 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [sys.catalog]: This master's current role is: LEARNER
I20260925 07:25:35.099141 12404 raft_consensus.cc:1050] T 00000000000000000000000000000000 P ea66681bd57f42b0a9db2845ec74d84d: attempting to promote NON_VOTER 87d6c9fb88664a3aa4245a34c108f564 to VOTER
I20260925 07:25:35.099298 12404 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea66681bd57f42b0a9db2845ec74d84d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ea66681bd57f42b0a9db2845ec74d84d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37781 } } peers { permanent_uuid: "87d6c9fb88664a3aa4245a34c108f564" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 44439 } attrs { promote: false } } peers { permanent_uuid: "52179093193647ce9b274a063c18597b" member_type: NON_VOTER last_known_addr { host: "127.5.158.124" port: 36829 } attrs { promote: true } }
I20260925 07:25:35.099713 12453 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [term 1 LEARNER]: Refusing update from remote peer ea66681bd57f42b0a9db2845ec74d84d: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260925 07:25:35.099807 12402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:35.099613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.124:36829 (local address 127.0.0.1:38458)
0925 07:25:35.099688 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:35.099742 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.124:36829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260925 07:25:35.099882 12403 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ea66681bd57f42b0a9db2845ec74d84d [LEADER]: Connected to new peer: Peer: permanent_uuid: "87d6c9fb88664a3aa4245a34c108f564" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 44439 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260925 07:25:35.100160 12402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:35.100086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.124:36829 (local address 127.0.0.1:38480)
0925 07:25:35.100107 (+ 21us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:35.100146 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.124:36829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260925 07:25:35.100307 11830 consensus_peers.cc:487] T 00000000000000000000000000000000 P ea66681bd57f42b0a9db2845ec74d84d -> Peer 52179093193647ce9b274a063c18597b (127.5.158.124:36829): Couldn't send request to peer 52179093193647ce9b274a063c18597b. Status: Network error: Client connection negotiation failed: client connection to 127.5.158.124:36829: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260925 07:25:35.100543 12404 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ea66681bd57f42b0a9db2845ec74d84d [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 87d6c9fb88664a3aa4245a34c108f564 (127.5.158.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ea66681bd57f42b0a9db2845ec74d84d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37781 } } peers { permanent_uuid: "87d6c9fb88664a3aa4245a34c108f564" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 44439 } attrs { promote: false } } peers { permanent_uuid: "52179093193647ce9b274a063c18597b" member_type: NON_VOTER last_known_addr { host: "127.5.158.124" port: 36829 } attrs { promote: true } } }
I20260925 07:25:35.100675 12453 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 87d6c9fb88664a3aa4245a34c108f564 (127.5.158.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ea66681bd57f42b0a9db2845ec74d84d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37781 } } peers { permanent_uuid: "87d6c9fb88664a3aa4245a34c108f564" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 44439 } attrs { promote: false } } peers { permanent_uuid: "52179093193647ce9b274a063c18597b" member_type: NON_VOTER last_known_addr { host: "127.5.158.124" port: 36829 } attrs { promote: true } } }
I20260925 07:25:35.101275 12404 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea66681bd57f42b0a9db2845ec74d84d [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ea66681bd57f42b0a9db2845ec74d84d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ea66681bd57f42b0a9db2845ec74d84d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37781 } } peers { permanent_uuid: "87d6c9fb88664a3aa4245a34c108f564" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 44439 } attrs { promote: false } } peers { permanent_uuid: "52179093193647ce9b274a063c18597b" member_type: NON_VOTER last_known_addr { host: "127.5.158.124" port: 36829 } attrs { promote: true } } }
I20260925 07:25:35.101315 12405 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea66681bd57f42b0a9db2845ec74d84d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ea66681bd57f42b0a9db2845ec74d84d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ea66681bd57f42b0a9db2845ec74d84d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37781 } } peers { permanent_uuid: "87d6c9fb88664a3aa4245a34c108f564" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 44439 } attrs { promote: false } } peers { permanent_uuid: "52179093193647ce9b274a063c18597b" member_type: NON_VOTER last_known_addr { host: "127.5.158.124" port: 36829 } attrs { promote: true } } }
I20260925 07:25:35.101349 12404 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea66681bd57f42b0a9db2845ec74d84d [sys.catalog]: This master's current role is: LEADER
I20260925 07:25:35.101364 12405 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea66681bd57f42b0a9db2845ec74d84d [sys.catalog]: This master's current role is: LEADER
I20260925 07:25:35.101468 12482 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ea66681bd57f42b0a9db2845ec74d84d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ea66681bd57f42b0a9db2845ec74d84d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37781 } } peers { permanent_uuid: "87d6c9fb88664a3aa4245a34c108f564" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 44439 } attrs { promote: false } } peers { permanent_uuid: "52179093193647ce9b274a063c18597b" member_type: NON_VOTER last_known_addr { host: "127.5.158.124" port: 36829 } attrs { promote: true } } }
I20260925 07:25:35.101539 12482 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 07:25:35.107183 12497 master.cc:552] Master@127.5.158.124:36829 shutdown complete.
I20260925 07:25:35.107230 12497 master_runner.cc:305] Clearing existing system tablet
I20260925 07:25:35.107350 12497 fs_manager.cc:359] Metadata directory not provided
I20260925 07:25:35.107383 12497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:25:35.107951 12497 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:25:35.108381 12594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:35.108486 12497 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:25:35.108539 12497 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:35.108562 12497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-2/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-2/wal
uuid: "52179093193647ce9b274a063c18597b"
format_stamp: "Formatted at 2026-09-25 07:25:35 on dist-test-slave-1drq"
server_key: "6f80e6d3f98ccd609a831f67d6db0355"
server_key_iv: "a51da98c8728dd99d941f68f1a427c37"
server_key_version: "encryptionkey@0"
I20260925 07:25:35.109156 12497 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 52179093193647ce9b274a063c18597b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:25:35.110229 12497 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 52179093193647ce9b274a063c18597b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260925 07:25:35.110278 12497 log.cc:1196] T 00000000000000000000000000000000 P 52179093193647ce9b274a063c18597b: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260925 07:25:35.110466 12497 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 52179093193647ce9b274a063c18597b: Deleting consensus metadata
I20260925 07:25:35.110623 12497 master_runner.cc:315] Copying system tablet from 127.5.158.126:37781
I20260925 07:25:35.111158 12497 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 52179093193647ce9b274a063c18597b: tablet copy: Beginning tablet copy session from remote peer at address 127.5.158.126:37781
I20260925 07:25:35.116355 11903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:35.111455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59438 (local address 127.5.158.126:37781)
0925 07:25:35.111488 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:35.111490 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:35.111569 (+ 79us) server_negotiation.cc:407] Connection header received
0925 07:25:35.111637 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:35.111638 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:35.111647 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:35.111698 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:35.111975 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.112407 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.113046 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.113294 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.113367 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:35.114161 (+ 794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:35.114162 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:35.114162 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:35.114168 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:35.115378 (+ 1210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:35.115726 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:35.115727 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:35.115728 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:35.115775 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:35.115947 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:35.115948 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:35.115948 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:35.116043 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:35.116106 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:35.116219 (+ 113us) server_negotiation.cc:299] Negotiation successful
0925 07:25:35.116281 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260925 07:25:35.116376 12600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:35.111412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.0.0.1:59438)
0925 07:25:35.111516 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:35.111526 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:35.111583 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:35.111703 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:35.111704 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:35.111846 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:35.111898 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:35.111899 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.112474 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.112476 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:35.112953 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:35.112954 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.113370 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.113371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:35.113431 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:35.113848 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:35.113852 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:35.114085 (+ 233us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:35.115470 (+ 1385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:35.115471 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:35.115472 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:35.115664 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:35.115836 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:35.115837 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:35.115837 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:35.115887 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:35.116099 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:35.116100 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:35.116137 (+ 37us) client_negotiation.cc:769] Sending connection context
0925 07:25:35.116195 (+ 58us) client_negotiation.cc:240] Negotiation successful
0925 07:25:35.116282 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":66,"threads_started":1}
I20260925 07:25:35.116601 11864 tablet_copy_service.cc:140] P ea66681bd57f42b0a9db2845ec74d84d: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 52179093193647ce9b274a063c18597b ({username='oryx', principal='oryx/127.5.158.124@KRBTEST.COM'} at 127.0.0.1:59438)
I20260925 07:25:35.116659 11864 tablet_copy_service.cc:161] P ea66681bd57f42b0a9db2845ec74d84d: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 52179093193647ce9b274a063c18597b at {username='oryx', principal='oryx/127.5.158.124@KRBTEST.COM'} at 127.0.0.1:59438: session id = 52179093193647ce9b274a063c18597b-00000000000000000000000000000000
I20260925 07:25:35.117079 11864 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P ea66681bd57f42b0a9db2845ec74d84d: Tablet Copy: opened 0 blocks and 1 log segments
I20260925 07:25:35.117522 12497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260925 07:25:35.118305 12497 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 52179093193647ce9b274a063c18597b: tablet copy: Starting download of 0 data blocks...
I20260925 07:25:35.118422 12497 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 52179093193647ce9b274a063c18597b: tablet copy: Starting download of 1 WAL segments...
I20260925 07:25:35.119019 12497 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 52179093193647ce9b274a063c18597b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260925 07:25:35.119616 11864 tablet_copy_service.cc:342] P ea66681bd57f42b0a9db2845ec74d84d: Request end of tablet copy session 52179093193647ce9b274a063c18597b-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.5.158.124@KRBTEST.COM'} at 127.0.0.1:59438
I20260925 07:25:35.119659 11864 tablet_copy_service.cc:433] P ea66681bd57f42b0a9db2845ec74d84d: ending tablet copy session 52179093193647ce9b274a063c18597b-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 52179093193647ce9b274a063c18597b
I20260925 07:25:35.121974 12497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:25:35.123057 12601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:35.123055 12604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:35.123246 12602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:35.123271 12497 server_base.cc:1034] running on GCE node
I20260925 07:25:35.123390 12497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:25:35.123528 12497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:25:35.124641 12497 hybrid_clock.cc:648] HybridClock initialized: now 1790321135124632 us; error 24 us; skew 500 ppm
Sep 25 07:25:35 dist-test-slave-1drq krb5kdc[11799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321135, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.158.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:25:35.125713 12497 init.cc:377] Logged in from keytab as oryx/127.5.158.124@KRBTEST.COM (short username oryx)
I20260925 07:25:35.125941 12497 webserver.cc:460] Webserver started at http://127.5.158.124:40289/ using document root <none> and password file <none>
I20260925 07:25:35.126086 12497 fs_manager.cc:359] Metadata directory not provided
I20260925 07:25:35.126124 12497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:25:35.126655 12497 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260925 07:25:35.127177 12611 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:35.127287 12497 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:35.127332 12497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-2/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-2/wal
uuid: "52179093193647ce9b274a063c18597b"
format_stamp: "Formatted at 2026-09-25 07:25:35 on dist-test-slave-1drq"
server_key: "6f80e6d3f98ccd609a831f67d6db0355"
server_key_iv: "a51da98c8728dd99d941f68f1a427c37"
server_key_version: "encryptionkey@0"
I20260925 07:25:35.127380 12497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:25:35.145530 12497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:25:35.146806 12497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:25:35.149672 12497 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.124:36829
I20260925 07:25:35.149685 12663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.124:36829 every 8 connection(s)
I20260925 07:25:35.149926 12497 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/master-2/data/info.pb
I20260925 07:25:35.150765 12664 sys_catalog.cc:263] Verifying existing consensus state
I20260925 07:25:35.151250 12664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52179093193647ce9b274a063c18597b: Bootstrap starting.
I20260925 07:25:35.154547 12664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52179093193647ce9b274a063c18597b: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 07:25:35.154742 12664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52179093193647ce9b274a063c18597b: Bootstrap complete.
I20260925 07:25:35.154897 12664 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52179093193647ce9b274a063c18597b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ea66681bd57f42b0a9db2845ec74d84d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37781 } } peers { permanent_uuid: "87d6c9fb88664a3aa4245a34c108f564" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 44439 } attrs { promote: false } } peers { permanent_uuid: "52179093193647ce9b274a063c18597b" member_type: NON_VOTER last_known_addr { host: "127.5.158.124" port: 36829 } attrs { promote: true } }
I20260925 07:25:35.154973 12664 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52179093193647ce9b274a063c18597b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 52179093193647ce9b274a063c18597b, State: Initialized, Role: LEARNER
I20260925 07:25:35.155007 12664 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52179093193647ce9b274a063c18597b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ea66681bd57f42b0a9db2845ec74d84d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37781 } } peers { permanent_uuid: "87d6c9fb88664a3aa4245a34c108f564" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 44439 } attrs { promote: false } } peers { permanent_uuid: "52179093193647ce9b274a063c18597b" member_type: NON_VOTER last_known_addr { host: "127.5.158.124" port: 36829 } attrs { promote: true } }
I20260925 07:25:35.155145 12664 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52179093193647ce9b274a063c18597b [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:25:35.156314 12668 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52179093193647ce9b274a063c18597b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ea66681bd57f42b0a9db2845ec74d84d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37781 } } peers { permanent_uuid: "87d6c9fb88664a3aa4245a34c108f564" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 44439 } attrs { promote: false } } peers { permanent_uuid: "52179093193647ce9b274a063c18597b" member_type: NON_VOTER last_known_addr { host: "127.5.158.124" port: 36829 } attrs { promote: true } } }
I20260925 07:25:35.157222 12666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:35.150271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38484 (local address 127.5.158.124:36829)
0925 07:25:35.150457 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:35.150459 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:35.150464 (+ 5us) server_negotiation.cc:407] Connection header received
0925 07:25:35.150479 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:35.150480 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:35.150487 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:35.150555 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:35.151205 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.151950 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.152745 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.152937 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.153003 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:35.154018 (+ 1015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:35.154019 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:35.154019 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:35.154025 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:35.155481 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:35.155860 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:35.155861 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:35.155862 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:35.155914 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:35.156108 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:35.156109 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:35.156109 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:35.156223 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:35.156320 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:35.156750 (+ 430us) server_negotiation.cc:299] Negotiation successful
0925 07:25:35.156826 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":68,"threads_started":1}
I20260925 07:25:35.156807 12668 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52179093193647ce9b274a063c18597b [sys.catalog]: This master's current role is: LEARNER
I20260925 07:25:35.156735 12679 catalog_manager.cc:1259] Loaded cluster ID: c10bd3f7239e4d6aaf544a03c0ac2c01
I20260925 07:25:35.157357 12679 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 52179093193647ce9b274a063c18597b: loading cluster ID for follower catalog manager: success
I20260925 07:25:35.157640 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 11906
I20260925 07:25:35.159061 12679 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 52179093193647ce9b274a063c18597b: acquiring CA information for follower catalog manager: success
I20260925 07:25:35.159349 12679 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 52179093193647ce9b274a063c18597b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 07:25:35.162357 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.65:41917
--local_ip_for_outbound_sockets=127.5.158.65
--tserver_master_addrs=127.5.158.126:37781
--webserver_port=41183
--webserver_interface=127.5.158.65
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/oryx_127.5.158.65.keytab
--principal=oryx/127.5.158.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:45003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.5.158.126:37781,127.5.158.125:44439
--tserver_master_addrs=127.5.158.126:37781,127.5.158.125:44439,127.5.158.124:36829 with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:25:35.234438 12680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:25:35.234599 12680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:25:35.234619 12680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:25:35.235620 12680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:25:35.235656 12680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:25:35.235680 12680 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.65
I20260925 07:25:35.237002 12680 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:45003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/oryx_127.5.158.65.keytab
--principal=oryx/127.5.158.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.65:41917
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.158.65
--webserver_port=41183
--tserver_master_addrs=127.5.158.126:37781,127.5.158.125:44439,127.5.158.124:36829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12680
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.65
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:25:35.237228 12680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:25:35.237447 12680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:25:35.241791 12685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:35.241900 12680 server_base.cc:1034] running on GCE node
W20260925 07:25:35.241833 12688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:35.241779 12686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:35.242202 12680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:25:35.242399 12680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:25:35.243621 12680 hybrid_clock.cc:648] HybridClock initialized: now 1790321135243595 us; error 25 us; skew 500 ppm
Sep 25 07:25:35 dist-test-slave-1drq krb5kdc[11799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321135, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.158.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:25:35.245873 12680 init.cc:377] Logged in from keytab as oryx/127.5.158.65@KRBTEST.COM (short username oryx)
I20260925 07:25:35.246259 12680 webserver.cc:460] Webserver started at http://127.5.158.65:41183/ using document root <none> and password file <none>
I20260925 07:25:35.246436 12680 fs_manager.cc:359] Metadata directory not provided
I20260925 07:25:35.246483 12680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:25:35.247653 12680 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 07:25:35.248183 12695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:35.248359 12680 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 07:25:35.248456 12680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-0/wal
uuid: "633f6da4f11c4910becf8aa67dc147c9"
format_stamp: "Formatted at 2026-09-25 07:25:34 on dist-test-slave-1drq"
server_key: "76fc65242c36c6d69c472fdcc93351c0"
server_key_iv: "7cf0dfedeffbfb7b4973467cfa0123ed"
server_key_version: "encryptionkey@0"
I20260925 07:25:35.248718 12680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:25:35.267249 12680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:25:35.268810 12680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:25:35.268896 12680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:25:35.269019 12680 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:25:35.269259 12680 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:25:35.269286 12680 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:35.269302 12680 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:25:35.269312 12680 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:35.274670 12680 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.65:41917
I20260925 07:25:35.274701 12808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.65:41917 every 8 connection(s)
I20260925 07:25:35.275043 12680 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
I20260925 07:25:35.276542 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 12680
I20260925 07:25:35.276703 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 12040
Sep 25 07:25:35 dist-test-slave-1drq krb5kdc[11799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321135, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.158.65@KRBTEST.COM for oryx/127.5.158.124@KRBTEST.COM
Sep 25 07:25:35 dist-test-slave-1drq krb5kdc[11799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321135, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.158.65@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM
Sep 25 07:25:35 dist-test-slave-1drq krb5kdc[11799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321135, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.158.65@KRBTEST.COM for oryx/127.5.158.125@KRBTEST.COM
I20260925 07:25:35.283923 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.66:32935
--local_ip_for_outbound_sockets=127.5.158.66
--tserver_master_addrs=127.5.158.126:37781
--webserver_port=34853
--webserver_interface=127.5.158.66
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/oryx_127.5.158.66.keytab
--principal=oryx/127.5.158.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:45003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.5.158.126:37781,127.5.158.125:44439
--tserver_master_addrs=127.5.158.126:37781,127.5.158.125:44439,127.5.158.124:36829 with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 07:25:35.286432 11903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:35.275622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:55701 (local address 127.5.158.126:37781)
0925 07:25:35.275667 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:35.275669 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:35.276110 (+ 441us) server_negotiation.cc:407] Connection header received
0925 07:25:35.276482 (+ 372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:35.276484 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:35.276497 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:35.276529 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:35.277401 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.278228 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.279640 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.279954 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.280001 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:35.283181 (+ 3180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:35.283182 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:35.283182 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:35.283189 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:35.284685 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:35.285388 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:35.285389 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:35.285390 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:35.285449 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:35.285715 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:35.285716 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:35.285717 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:35.285838 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:35.285885 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:35.286234 (+ 349us) server_negotiation.cc:299] Negotiation successful
0925 07:25:35.286310 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 07:25:35.286453 12494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:35.275623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:35199 (local address 127.5.158.125:44439)
0925 07:25:35.275672 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:35.275674 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:35.275938 (+ 264us) server_negotiation.cc:407] Connection header received
0925 07:25:35.276453 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:35.276454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:35.276465 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:35.276517 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:35.277365 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.278146 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.279617 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.279891 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.279928 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:35.283628 (+ 3700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:35.283629 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:35.283629 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:35.283636 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:35.285297 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:35.285640 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:35.285641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:35.285641 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:35.285692 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:35.285876 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:35.285877 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:35.285877 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:35.286010 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:35.286062 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:35.286275 (+ 213us) server_negotiation.cc:299] Negotiation successful
0925 07:25:35.286349 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260925 07:25:35.286919 12815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:35.275641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.65:55701)
0925 07:25:35.276053 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:35.276063 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:35.276385 (+ 322us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:35.276628 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:35.276633 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:35.277119 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:35.277304 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:35.277306 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.278319 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.278320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:35.279527 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:35.279530 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.280069 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.280069 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:35.280149 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:35.281033 (+ 884us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:35.281038 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:35.283079 (+ 2041us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:35.284753 (+ 1674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:35.284759 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:35.284760 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:35.285294 (+ 534us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:35.285532 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:35.285532 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:35.285533 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:35.285649 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:35.285915 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:35.285920 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:35.286086 (+ 166us) client_negotiation.cc:769] Sending connection context
0925 07:25:35.286525 (+ 439us) client_negotiation.cc:240] Negotiation successful
0925 07:25:35.286684 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377,"mutex_wait_us":70,"thread_start_us":103,"threads_started":1}
I20260925 07:25:35.286990 12666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:35.276110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:38919 (local address 127.5.158.124:36829)
0925 07:25:35.276150 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:35.276151 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:35.276576 (+ 425us) server_negotiation.cc:407] Connection header received
0925 07:25:35.276727 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:35.276728 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:35.276742 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:35.276771 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:35.277599 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.278315 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.279617 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.279891 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.279928 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:35.283890 (+ 3962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:35.283891 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:35.283892 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:35.283897 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:35.285882 (+ 1985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:35.286236 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:35.286237 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:35.286237 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:35.286285 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:35.286456 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:35.286457 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:35.286457 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:35.286575 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:35.286620 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:35.286817 (+ 197us) server_negotiation.cc:299] Negotiation successful
0925 07:25:35.286889 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 07:25:35.287385 12810 heartbeater.cc:347] Connected to a master server at 127.5.158.126:37781
I20260925 07:25:35.287503 12810 heartbeater.cc:464] Registering TS with master...
I20260925 07:25:35.287519 12814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:35.275641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.125:44439 (local address 127.5.158.65:35199)
0925 07:25:35.275812 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:35.275840 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:35.276362 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:35.276628 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:35.276632 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:35.276995 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:35.277261 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:35.277266 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.278269 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.278269 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:35.279523 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:35.279525 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.280007 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.280008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:35.280146 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:35.280992 (+ 846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:35.280995 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:35.283537 (+ 2542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:35.285380 (+ 1843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:35.285381 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:35.285381 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:35.285563 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:35.285760 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:35.285761 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:35.285761 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:35.285814 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:35.286104 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:35.286104 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:35.286156 (+ 52us) client_negotiation.cc:769] Sending connection context
0925 07:25:35.286841 (+ 685us) client_negotiation.cc:240] Negotiation successful
0925 07:25:35.286930 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":107,"thread_start_us":102,"threads_started":1}
I20260925 07:25:35.287694 12810 heartbeater.cc:511] Master 127.5.158.126:37781 requested a full tablet report, sending...
I20260925 07:25:35.288105 12813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:35.275922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.124:36829 (local address 127.5.158.65:38919)
0925 07:25:35.276523 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:35.276532 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:35.276646 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:35.277300 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:35.277301 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:35.277458 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:35.277508 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:35.277509 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.278387 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.278388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:35.279523 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:35.279525 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.280009 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.280009 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:35.280153 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:35.280955 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:35.280965 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:35.283283 (+ 2318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:35.285951 (+ 2668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:35.285952 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:35.285952 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:35.286159 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:35.286351 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:35.286351 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:35.286352 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:35.286397 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:35.286683 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:35.286684 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:35.286722 (+ 38us) client_negotiation.cc:769] Sending connection context
0925 07:25:35.287097 (+ 375us) client_negotiation.cc:240] Negotiation successful
0925 07:25:35.287189 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"mutex_wait_us":81,"thread_start_us":71,"threads_started":1}
I20260925 07:25:35.288115 11844 ts_manager.cc:194] Re-registered known tserver with Master: 633f6da4f11c4910becf8aa67dc147c9 (127.5.158.65:41917)
I20260925 07:25:35.288431 12809 heartbeater.cc:347] Connected to a master server at 127.5.158.124:36829
I20260925 07:25:35.288473 12809 heartbeater.cc:464] Registering TS with master...
I20260925 07:25:35.288532 12809 heartbeater.cc:511] Master 127.5.158.124:36829 requested a full tablet report, sending...
I20260925 07:25:35.288594 11844 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.158.65@KRBTEST.COM'} at 127.5.158.65:55701
I20260925 07:25:35.288727 12811 heartbeater.cc:347] Connected to a master server at 127.5.158.125:44439
I20260925 07:25:35.288770 12811 heartbeater.cc:464] Registering TS with master...
I20260925 07:25:35.288825 12811 heartbeater.cc:511] Master 127.5.158.125:44439 requested a full tablet report, sending...
I20260925 07:25:35.288856 12628 ts_manager.cc:194] Registered new tserver with Master: 633f6da4f11c4910becf8aa67dc147c9 (127.5.158.65:41917)
I20260925 07:25:35.289179 12443 ts_manager.cc:194] Registered new tserver with Master: 633f6da4f11c4910becf8aa67dc147c9 (127.5.158.65:41917)
W20260925 07:25:35.357810 12816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:25:35.357991 12816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:25:35.358014 12816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:25:35.359093 12816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:25:35.359129 12816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:25:35.359153 12816 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.66
I20260925 07:25:35.360741 12816 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:45003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/oryx_127.5.158.66.keytab
--principal=oryx/127.5.158.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.66:32935
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.158.66
--webserver_port=34853
--tserver_master_addrs=127.5.158.126:37781,127.5.158.125:44439,127.5.158.124:36829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12816
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.66
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:25:35.360968 12816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:25:35.361166 12816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:25:35.366637 12821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:35.366642 12822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:35.366752 12824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:35.366655 12816 server_base.cc:1034] running on GCE node
I20260925 07:25:35.366986 12816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:25:35.367215 12816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:25:35.368355 12816 hybrid_clock.cc:648] HybridClock initialized: now 1790321135368341 us; error 25 us; skew 500 ppm
Sep 25 07:25:35 dist-test-slave-1drq krb5kdc[11799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321135, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.158.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:25:35.370337 12816 init.cc:377] Logged in from keytab as oryx/127.5.158.66@KRBTEST.COM (short username oryx)
I20260925 07:25:35.370695 12816 webserver.cc:460] Webserver started at http://127.5.158.66:34853/ using document root <none> and password file <none>
I20260925 07:25:35.370883 12816 fs_manager.cc:359] Metadata directory not provided
I20260925 07:25:35.370951 12816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:25:35.372068 12816 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 07:25:35.372555 12831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:35.372794 12816 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:35.372872 12816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-1/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-1/wal
uuid: "2bcb44a2433144509be8441477797a48"
format_stamp: "Formatted at 2026-09-25 07:25:34 on dist-test-slave-1drq"
server_key: "b7512700d5dab35d6ecefbb430246152"
server_key_iv: "abca6e360c20d137c88d7c1ca73f7028"
server_key_version: "encryptionkey@0"
I20260925 07:25:35.373174 12816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:25:35.383873 12816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:25:35.385691 12816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:25:35.385864 12816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:25:35.386029 12816 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:25:35.386226 12816 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:25:35.386255 12816 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:35.386272 12816 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:25:35.386289 12816 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:35.391634 12816 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.66:32935
I20260925 07:25:35.391728 12944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.66:32935 every 8 connection(s)
I20260925 07:25:35.392005 12816 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
Sep 25 07:25:35 dist-test-slave-1drq krb5kdc[11799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321135, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.158.66@KRBTEST.COM for oryx/127.5.158.124@KRBTEST.COM
I20260925 07:25:35.398999 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 12816
I20260925 07:25:35.399142 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 12174
Sep 25 07:25:35 dist-test-slave-1drq krb5kdc[11799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321135, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.158.66@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM
Sep 25 07:25:35 dist-test-slave-1drq krb5kdc[11799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321135, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.158.66@KRBTEST.COM for oryx/127.5.158.125@KRBTEST.COM
I20260925 07:25:35.402258 12666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:35.392556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:35223 (local address 127.5.158.124:36829)
0925 07:25:35.392604 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:35.392606 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:35.392809 (+ 203us) server_negotiation.cc:407] Connection header received
0925 07:25:35.393980 (+ 1171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:35.393981 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:35.393994 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:35.394026 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:35.394796 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.395540 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.396615 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.396802 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.396883 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:35.399624 (+ 2741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:35.399635 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:35.399635 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:35.399641 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:35.401005 (+ 1364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:35.401424 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:35.401425 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:35.401426 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:35.401481 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:35.401712 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:35.401713 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:35.401714 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:35.401815 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:35.401881 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:35.402116 (+ 235us) server_negotiation.cc:299] Negotiation successful
0925 07:25:35.402180 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 07:25:35.402411 12950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:35.392560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.124:36829 (local address 127.5.158.66:35223)
0925 07:25:35.392716 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:35.392742 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:35.393888 (+ 1146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:35.394124 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:35.394133 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:35.394506 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:35.394664 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:35.394670 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.395632 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.395632 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:35.396504 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:35.396505 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.396979 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.396980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:35.397065 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:35.397726 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:35.397731 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:35.399147 (+ 1416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:35.401099 (+ 1952us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:35.401105 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:35.401106 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:35.401354 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:35.401544 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:35.401544 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:35.401544 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:35.401639 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:35.401874 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:35.401880 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:35.401996 (+ 116us) client_negotiation.cc:769] Sending connection context
0925 07:25:35.402089 (+ 93us) client_negotiation.cc:240] Negotiation successful
0925 07:25:35.402224 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":67,"thread_start_us":106,"threads_started":1}
I20260925 07:25:35.402879 12945 heartbeater.cc:347] Connected to a master server at 127.5.158.124:36829
I20260925 07:25:35.402956 12945 heartbeater.cc:464] Registering TS with master...
I20260925 07:25:35.403105 12945 heartbeater.cc:511] Master 127.5.158.124:36829 requested a full tablet report, sending...
I20260925 07:25:35.403380 11903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:35.392804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:59545 (local address 127.5.158.126:37781)
0925 07:25:35.392844 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:35.392845 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:35.392853 (+ 8us) server_negotiation.cc:407] Connection header received
0925 07:25:35.393994 (+ 1141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:35.393996 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:35.394009 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:35.394070 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:35.394754 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.395236 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.396364 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.396640 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.396695 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:35.400533 (+ 3838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:35.400534 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:35.400534 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:35.400541 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:35.402204 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:35.402680 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:35.402681 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:35.402681 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:35.402731 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:35.402917 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:35.402918 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:35.402918 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:35.403046 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:35.403114 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:35.403255 (+ 141us) server_negotiation.cc:299] Negotiation successful
0925 07:25:35.403328 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 07:25:35.403465 12949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:35.392560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.66:59545)
0925 07:25:35.392707 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:35.392742 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:35.393875 (+ 1133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:35.394124 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:35.394128 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:35.394486 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:35.394664 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:35.394668 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.395391 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.395391 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:35.396210 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:35.396212 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.396749 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.396750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:35.397054 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:35.397646 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:35.397656 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:35.400233 (+ 2577us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:35.402282 (+ 2049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:35.402283 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:35.402283 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:35.402603 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:35.402793 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:35.402794 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:35.402794 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:35.402845 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:35.403116 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:35.403117 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:35.403159 (+ 42us) client_negotiation.cc:769] Sending connection context
0925 07:25:35.403228 (+ 69us) client_negotiation.cc:240] Negotiation successful
0925 07:25:35.403323 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":84,"threads_started":1}
I20260925 07:25:35.403743 12628 ts_manager.cc:194] Registered new tserver with Master: 2bcb44a2433144509be8441477797a48 (127.5.158.66:32935)
I20260925 07:25:35.405069 12946 heartbeater.cc:347] Connected to a master server at 127.5.158.126:37781
I20260925 07:25:35.405138 12946 heartbeater.cc:464] Registering TS with master...
I20260925 07:25:35.405194 12946 heartbeater.cc:511] Master 127.5.158.126:37781 requested a full tablet report, sending...
I20260925 07:25:35.405437 11844 ts_manager.cc:194] Re-registered known tserver with Master: 2bcb44a2433144509be8441477797a48 (127.5.158.66:32935)
I20260925 07:25:35.405499 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.67:44005
--local_ip_for_outbound_sockets=127.5.158.67
--tserver_master_addrs=127.5.158.126:37781
--webserver_port=35851
--webserver_interface=127.5.158.67
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/oryx_127.5.158.67.keytab
--principal=oryx/127.5.158.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:45003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.5.158.126:37781,127.5.158.125:44439
--tserver_master_addrs=127.5.158.126:37781,127.5.158.125:44439,127.5.158.124:36829 with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 07:25:35.405906 11844 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.158.66@KRBTEST.COM'} at 127.5.158.66:59545
I20260925 07:25:35.408264 12494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:35.392968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:54867 (local address 127.5.158.125:44439)
0925 07:25:35.393014 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:35.393016 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:35.393047 (+ 31us) server_negotiation.cc:407] Connection header received
0925 07:25:35.394120 (+ 1073us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:35.394121 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:35.394131 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:35.394181 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:35.394764 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.395490 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.396586 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.396831 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.396889 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:35.405417 (+ 8528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:35.405418 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:35.405419 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:35.405426 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:35.407136 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:35.407466 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:35.407468 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:35.407468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:35.407531 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:35.407726 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:35.407727 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:35.407727 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:35.407866 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:35.407933 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:35.408028 (+ 95us) server_negotiation.cc:299] Negotiation successful
0925 07:25:35.408108 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 07:25:35.408444 12951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:35.392813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.125:44439 (local address 127.5.158.66:54867)
0925 07:25:35.392997 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:35.393005 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:35.393888 (+ 883us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:35.394194 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:35.394195 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:35.394502 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:35.394665 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:35.394670 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.395584 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.395585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:35.396502 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:35.396504 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.396954 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.396955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:35.397054 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:35.397661 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:35.397665 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:35.405340 (+ 7675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:35.407233 (+ 1893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:35.407234 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:35.407235 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:35.407398 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:35.407600 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:35.407601 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:35.407601 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:35.407662 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:35.407927 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:35.407927 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:35.407957 (+ 30us) client_negotiation.cc:769] Sending connection context
0925 07:25:35.408006 (+ 49us) client_negotiation.cc:240] Negotiation successful
0925 07:25:35.408078 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":25,"thread_start_us":61,"threads_started":1}
I20260925 07:25:35.408814 12947 heartbeater.cc:347] Connected to a master server at 127.5.158.125:44439
I20260925 07:25:35.408866 12947 heartbeater.cc:464] Registering TS with master...
I20260925 07:25:35.408931 12947 heartbeater.cc:511] Master 127.5.158.125:44439 requested a full tablet report, sending...
I20260925 07:25:35.409225 12443 ts_manager.cc:194] Registered new tserver with Master: 2bcb44a2433144509be8441477797a48 (127.5.158.66:32935)
W20260925 07:25:35.480780 12952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:25:35.480952 12952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:25:35.480974 12952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:25:35.482031 12952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:25:35.482067 12952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:25:35.482090 12952 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.67
I20260925 07:25:35.483326 12952 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:45003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/krb5kdc/oryx_127.5.158.67.keytab
--principal=oryx/127.5.158.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.67:44005
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.158.67
--webserver_port=35851
--tserver_master_addrs=127.5.158.126:37781,127.5.158.125:44439,127.5.158.124:36829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12952
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.67
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:25:35.483487 12952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:25:35.483631 12952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:25:35.487950 12958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:35.487958 12957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:25:35.487958 12960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:25:35.488159 12952 server_base.cc:1034] running on GCE node
I20260925 07:25:35.488348 12952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:25:35.488562 12952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:25:35.489686 12952 hybrid_clock.cc:648] HybridClock initialized: now 1790321135489682 us; error 30 us; skew 500 ppm
Sep 25 07:25:35 dist-test-slave-1drq krb5kdc[11799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321135, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.158.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:25:35.491748 12952 init.cc:377] Logged in from keytab as oryx/127.5.158.67@KRBTEST.COM (short username oryx)
I20260925 07:25:35.492161 12952 webserver.cc:460] Webserver started at http://127.5.158.67:35851/ using document root <none> and password file <none>
I20260925 07:25:35.492353 12952 fs_manager.cc:359] Metadata directory not provided
I20260925 07:25:35.492416 12952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:25:35.493634 12952 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 07:25:35.494125 12967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:35.494275 12952 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:35.494349 12952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-2/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-2/wal
uuid: "470984d5f90d4994b15fdb6d0d13e6c6"
format_stamp: "Formatted at 2026-09-25 07:25:34 on dist-test-slave-1drq"
server_key: "d3b9bbae6d4630f47eb64a090c7c2c49"
server_key_iv: "64b1e487e0803b4133a689d1411c4804"
server_key_version: "encryptionkey@0"
I20260925 07:25:35.494602 12952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:25:35.501361 12952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:25:35.502789 12952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:25:35.502918 12952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:25:35.503101 12952 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:25:35.503338 12952 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:25:35.503366 12952 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:35.503398 12952 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:25:35.503422 12952 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:25:35.508710 12952 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.67:44005
I20260925 07:25:35.508759 13080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.67:44005 every 8 connection(s)
I20260925 07:25:35.509049 12952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
I20260925 07:25:35.510156 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 12952
Sep 25 07:25:35 dist-test-slave-1drq krb5kdc[11799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321135, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.158.67@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM
Sep 25 07:25:35 dist-test-slave-1drq krb5kdc[11799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321135, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.158.67@KRBTEST.COM for oryx/127.5.158.124@KRBTEST.COM
Sep 25 07:25:35 dist-test-slave-1drq krb5kdc[11799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321135, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.158.67@KRBTEST.COM for oryx/127.5.158.125@KRBTEST.COM
I20260925 07:25:35.519631 11903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:35.509754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:34007 (local address 127.5.158.126:37781)
0925 07:25:35.509793 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:35.509795 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:35.509938 (+ 143us) server_negotiation.cc:407] Connection header received
0925 07:25:35.510456 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:35.510458 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:35.510471 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:35.510524 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:35.511331 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.512019 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.513516 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.513795 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.513833 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:35.516500 (+ 2667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:35.516501 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:35.516501 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:35.516507 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:35.518071 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:35.518567 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:35.518568 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:35.518568 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:35.518622 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:35.518918 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:35.518919 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:35.518920 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:35.519047 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:35.519089 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:35.519433 (+ 344us) server_negotiation.cc:299] Negotiation successful
0925 07:25:35.519521 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 07:25:35.520298 13085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:35.509659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.67:34007)
0925 07:25:35.509816 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:35.509850 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:35.510371 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:35.510580 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:35.510583 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:35.511154 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:35.511226 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:35.511238 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.512125 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.512125 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:35.513431 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:35.513433 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.513910 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.513911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:35.514045 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:35.514576 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:35.514585 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:35.516398 (+ 1813us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:35.518175 (+ 1777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:35.518202 (+ 27us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:35.518204 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:35.518470 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:35.518689 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:35.518690 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:35.518690 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:35.518828 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:35.519136 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:35.519141 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:35.519285 (+ 144us) client_negotiation.cc:769] Sending connection context
0925 07:25:35.519902 (+ 617us) client_negotiation.cc:240] Negotiation successful
0925 07:25:35.520083 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":55,"thread_start_us":94,"threads_started":1}
I20260925 07:25:35.520777 13082 heartbeater.cc:347] Connected to a master server at 127.5.158.126:37781
I20260925 07:25:35.520850 13082 heartbeater.cc:464] Registering TS with master...
I20260925 07:25:35.520958 13093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:35.511296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42796 (local address 127.5.158.125:44439)
0925 07:25:35.511793 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:35.511795 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:35.511803 (+ 8us) server_negotiation.cc:407] Connection header received
0925 07:25:35.511879 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:35.511880 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:35.511891 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:35.511936 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:35.512664 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.513359 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.514369 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.514617 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.514677 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:35.516174 (+ 1497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:35.516175 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:35.516175 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:35.516182 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:35.518875 (+ 2693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:35.519318 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:35.519319 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:35.519320 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:35.519377 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:35.519566 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:35.519567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:35.519567 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:35.519793 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:35.519874 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:35.520233 (+ 359us) server_negotiation.cc:299] Negotiation successful
0925 07:25:35.520550 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":479,"thread_start_us":58,"threads_started":1}
I20260925 07:25:35.521003 13082 heartbeater.cc:511] Master 127.5.158.126:37781 requested a full tablet report, sending...
I20260925 07:25:35.521435 11844 ts_manager.cc:194] Re-registered known tserver with Master: 470984d5f90d4994b15fdb6d0d13e6c6 (127.5.158.67:44005)
I20260925 07:25:35.521907 11844 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.158.67@KRBTEST.COM'} at 127.5.158.67:34007
I20260925 07:25:35.523185 12494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:35.510019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:52463 (local address 127.5.158.125:44439)
0925 07:25:35.510059 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:35.510061 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:35.511907 (+ 1846us) server_negotiation.cc:407] Connection header received
0925 07:25:35.512262 (+ 355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:35.512263 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:35.512272 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:35.512298 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:35.513294 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.513953 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.516654 (+ 2701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.516930 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.516987 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:35.520238 (+ 3251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:35.520239 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:35.520239 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:35.520245 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:35.521698 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:35.522145 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:35.522146 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:35.522146 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:35.522210 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:35.522467 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:35.522468 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:35.522468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:35.522663 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:35.522725 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:35.523010 (+ 285us) server_negotiation.cc:299] Negotiation successful
0925 07:25:35.523096 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 07:25:35.523380 13086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:35.509660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.124:36829 (local address 127.5.158.67:34447)
0925 07:25:35.509836 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:35.509851 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:35.510365 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:35.510584 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:35.510584 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:35.510940 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:35.511206 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:35.511213 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.512124 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.512124 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:35.513429 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:35.513430 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.513938 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.513939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:35.514047 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:35.514689 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:35.514693 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:35.519744 (+ 5051us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:35.522211 (+ 2467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:35.522212 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:35.522212 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:35.522473 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:35.522709 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:35.522710 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:35.522710 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:35.522774 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:35.523052 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:35.523053 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:35.523097 (+ 44us) client_negotiation.cc:769] Sending connection context
0925 07:25:35.523170 (+ 73us) client_negotiation.cc:240] Negotiation successful
0925 07:25:35.523263 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":175,"spinlock_wait_cycles":4864,"thread_start_us":83,"threads_started":1}
I20260925 07:25:35.523389 12666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:35.509656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:34447 (local address 127.5.158.124:36829)
0925 07:25:35.509709 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:35.509711 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:35.509930 (+ 219us) server_negotiation.cc:407] Connection header received
0925 07:25:35.510484 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:35.510485 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:35.510497 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:35.510524 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:35.511315 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.512005 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.513512 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.513803 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.513854 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:35.519817 (+ 5963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:35.519818 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:35.519818 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:35.519824 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:35.521896 (+ 2072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:35.522557 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:35.522558 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:35.522558 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:35.522629 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:35.522843 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:35.522845 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:35.522845 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:35.522962 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:35.523004 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:35.523206 (+ 202us) server_negotiation.cc:299] Negotiation successful
0925 07:25:35.523284 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260925 07:25:35.523731 13081 heartbeater.cc:347] Connected to a master server at 127.5.158.124:36829
I20260925 07:25:35.523797 13081 heartbeater.cc:464] Registering TS with master...
I20260925 07:25:35.523866 13081 heartbeater.cc:511] Master 127.5.158.124:36829 requested a full tablet report, sending...
I20260925 07:25:35.524045 13087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:35.509917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.125:44439 (local address 127.5.158.67:52463)
0925 07:25:35.511856 (+ 1939us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:35.511864 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:35.512192 (+ 328us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:35.512999 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:35.512999 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:35.513168 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:25:35.513215 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:35.513216 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.514029 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.514030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:35.514836 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:35.514838 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.517064 (+ 2226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.517065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:35.517159 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:35.517618 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:25:35.517624 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:25:35.520144 (+ 2520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:25:35.521835 (+ 1691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:35.521835 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:35.521836 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:35.522063 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:35.522323 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:25:35.522324 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:25:35.522324 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:25:35.522383 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:25:35.522785 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:25:35.522786 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:25:35.522865 (+ 79us) client_negotiation.cc:769] Sending connection context
0925 07:25:35.523356 (+ 491us) client_negotiation.cc:240] Negotiation successful
0925 07:25:35.523672 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1916,"mutex_wait_us":38,"thread_start_us":52,"threads_started":1}
I20260925 07:25:35.524130 12628 ts_manager.cc:194] Registered new tserver with Master: 470984d5f90d4994b15fdb6d0d13e6c6 (127.5.158.67:44005)
I20260925 07:25:35.524458 13083 heartbeater.cc:347] Connected to a master server at 127.5.158.125:44439
I20260925 07:25:35.524504 13083 heartbeater.cc:464] Registering TS with master...
I20260925 07:25:35.524538 13083 heartbeater.cc:511] Master 127.5.158.125:44439 requested a full tablet report, sending...
I20260925 07:25:35.524771 13096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:35.511600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38494 (local address 127.5.158.124:36829)
0925 07:25:35.511689 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:35.511691 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:35.512063 (+ 372us) server_negotiation.cc:407] Connection header received
0925 07:25:35.512082 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:35.512083 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:35.512095 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:35.512160 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:35.514926 (+ 2766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.515613 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.516603 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.516897 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.516976 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:35.519260 (+ 2284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:35.519261 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:35.519261 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:35.519267 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:35.522289 (+ 3022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:35.523788 (+ 1499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:35.523789 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:35.523789 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:35.523840 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:35.524083 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:35.524085 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:35.524085 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:35.524250 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:35.524382 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:35.524584 (+ 202us) server_negotiation.cc:299] Negotiation successful
0925 07:25:35.524672 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72,"thread_start_us":48,"threads_started":1}
I20260925 07:25:35.524791 12443 ts_manager.cc:194] Registered new tserver with Master: 470984d5f90d4994b15fdb6d0d13e6c6 (127.5.158.67:44005)
I20260925 07:25:35.526417 13094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:35.511334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59454 (local address 127.5.158.126:37781)
0925 07:25:35.512465 (+ 1131us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:35.512467 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:35.512474 (+ 7us) server_negotiation.cc:407] Connection header received
0925 07:25:35.512493 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:35.512494 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:35.512506 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:35.517539 (+ 5033us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:35.517574 (+ 35us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.518328 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.520699 (+ 2371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.520994 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.522003 (+ 1009us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:35.524009 (+ 2006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:35.524010 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:35.524010 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:35.524017 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:35.525621 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:35.525908 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:35.525908 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:35.525908 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:35.525952 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:35.526071 (+ 119us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:35.526072 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:35.526072 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:35.526159 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:35.526211 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:35.526310 (+ 99us) server_negotiation.cc:299] Negotiation successful
0925 07:25:35.526360 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1105,"thread_start_us":82,"threads_started":1}
I20260925 07:25:35.527935 11844 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59454:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 07:25:35.529408 11844 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 07:25:35.535769 13102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:35.532489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.66:32935 (local address 127.0.0.1:38282)
0925 07:25:35.532756 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:35.532766 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:35.532808 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:35.532962 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:35.532962 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:35.532963 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:25:35.533001 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:35.533002 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.534074 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.534075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:35.534684 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:35.534685 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.535553 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.535554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:35.535612 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:35.535617 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 07:25:35.535665 (+ 48us) client_negotiation.cc:240] Negotiation successful
0925 07:25:35.535702 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":62,"threads_started":1}
I20260925 07:25:35.536054 13106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:35.532576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38282 (local address 127.5.158.66:32935)
0925 07:25:35.532740 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:35.532743 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:35.532802 (+ 59us) server_negotiation.cc:407] Connection header received
0925 07:25:35.532865 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:35.532866 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:35.532879 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:35.532957 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:25:35.533056 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.534002 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.534763 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.535374 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.535453 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:35.535519 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:35.535928 (+ 409us) server_negotiation.cc:299] Negotiation successful
0925 07:25:35.535972 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":87,"threads_started":1}
I20260925 07:25:35.536134 12402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:35.532312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:41917 (local address 127.0.0.1:40342)
0925 07:25:35.532347 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:35.532356 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:35.532385 (+ 29us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:35.532690 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:35.532691 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:35.532692 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:25:35.532726 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:35.532727 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.533906 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.533907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:35.535006 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:35.535009 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.535892 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.535893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:35.535964 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:35.535967 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 07:25:35.535997 (+ 30us) client_negotiation.cc:240] Negotiation successful
0925 07:25:35.536030 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22,"mutex_wait_us":42}
I20260925 07:25:35.536019 13105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:35.532337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.67:44005 (local address 127.0.0.1:33592)
0925 07:25:35.532741 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:35.532750 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:35.532793 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:35.533134 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:35.533135 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:35.533137 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:25:35.533180 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:35.533181 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.533920 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.533921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:35.535231 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:35.535233 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.535812 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.535813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:35.535868 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:35.535872 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 07:25:35.535912 (+ 40us) client_negotiation.cc:240] Negotiation successful
0925 07:25:35.535949 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":385,"mutex_wait_us":13,"thread_start_us":76,"threads_started":1}
I20260925 07:25:35.536419 13104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:35.532392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40342 (local address 127.5.158.65:41917)
0925 07:25:35.532534 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:35.532537 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:35.532548 (+ 11us) server_negotiation.cc:407] Connection header received
0925 07:25:35.532582 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:35.532584 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:35.532598 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:35.532680 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:25:35.532807 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.533819 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.535085 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.535796 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.536074 (+ 278us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:35.536181 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:35.536207 (+ 26us) server_negotiation.cc:299] Negotiation successful
0925 07:25:35.536324 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":59,"threads_started":1}
I20260925 07:25:35.536470 13103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:35.532386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33592 (local address 127.5.158.67:44005)
0925 07:25:35.532895 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:35.532899 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:35.532909 (+ 10us) server_negotiation.cc:407] Connection header received
0925 07:25:35.532943 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:35.532945 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:35.532957 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:35.533019 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:25:35.533241 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.533841 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.535297 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.535732 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.535810 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:35.536080 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:35.536201 (+ 121us) server_negotiation.cc:299] Negotiation successful
0925 07:25:35.536312 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":487,"thread_start_us":68,"threads_started":1}
I20260925 07:25:35.537034 12743 tablet_service.cc:1467] Processing CreateTablet for tablet dadd68ad4eb24b49a6c44c852edee260 (DEFAULT_TABLE table=test-table [id=409e8fa7add84ccdbcfa6912405ebc24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:25:35.537034 12879 tablet_service.cc:1467] Processing CreateTablet for tablet dadd68ad4eb24b49a6c44c852edee260 (DEFAULT_TABLE table=test-table [id=409e8fa7add84ccdbcfa6912405ebc24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:25:35.537206 13015 tablet_service.cc:1467] Processing CreateTablet for tablet dadd68ad4eb24b49a6c44c852edee260 (DEFAULT_TABLE table=test-table [id=409e8fa7add84ccdbcfa6912405ebc24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:25:35.537369 12879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dadd68ad4eb24b49a6c44c852edee260. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:25:35.537370 12743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dadd68ad4eb24b49a6c44c852edee260. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:25:35.537550 13015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dadd68ad4eb24b49a6c44c852edee260. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:25:35.539861 13107 tablet_bootstrap.cc:492] T dadd68ad4eb24b49a6c44c852edee260 P 470984d5f90d4994b15fdb6d0d13e6c6: Bootstrap starting.
I20260925 07:25:35.540479 13107 tablet_bootstrap.cc:654] T dadd68ad4eb24b49a6c44c852edee260 P 470984d5f90d4994b15fdb6d0d13e6c6: Neither blocks nor log segments found. Creating new log.
I20260925 07:25:35.540733 13109 tablet_bootstrap.cc:492] T dadd68ad4eb24b49a6c44c852edee260 P 633f6da4f11c4910becf8aa67dc147c9: Bootstrap starting.
I20260925 07:25:35.540915 13107 log.cc:824] T dadd68ad4eb24b49a6c44c852edee260 P 470984d5f90d4994b15fdb6d0d13e6c6: Log is configured to *not* fsync() on all Append() calls
I20260925 07:25:35.541182 13108 tablet_bootstrap.cc:492] T dadd68ad4eb24b49a6c44c852edee260 P 2bcb44a2433144509be8441477797a48: Bootstrap starting.
I20260925 07:25:35.541327 13109 tablet_bootstrap.cc:654] T dadd68ad4eb24b49a6c44c852edee260 P 633f6da4f11c4910becf8aa67dc147c9: Neither blocks nor log segments found. Creating new log.
I20260925 07:25:35.541627 13109 log.cc:824] T dadd68ad4eb24b49a6c44c852edee260 P 633f6da4f11c4910becf8aa67dc147c9: Log is configured to *not* fsync() on all Append() calls
I20260925 07:25:35.541647 13107 tablet_bootstrap.cc:492] T dadd68ad4eb24b49a6c44c852edee260 P 470984d5f90d4994b15fdb6d0d13e6c6: No bootstrap required, opened a new log
I20260925 07:25:35.541713 13107 ts_tablet_manager.cc:1397] T dadd68ad4eb24b49a6c44c852edee260 P 470984d5f90d4994b15fdb6d0d13e6c6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260925 07:25:35.541775 13108 tablet_bootstrap.cc:654] T dadd68ad4eb24b49a6c44c852edee260 P 2bcb44a2433144509be8441477797a48: Neither blocks nor log segments found. Creating new log.
I20260925 07:25:35.542034 13108 log.cc:824] T dadd68ad4eb24b49a6c44c852edee260 P 2bcb44a2433144509be8441477797a48: Log is configured to *not* fsync() on all Append() calls
I20260925 07:25:35.542323 13109 tablet_bootstrap.cc:492] T dadd68ad4eb24b49a6c44c852edee260 P 633f6da4f11c4910becf8aa67dc147c9: No bootstrap required, opened a new log
I20260925 07:25:35.542362 13109 ts_tablet_manager.cc:1397] T dadd68ad4eb24b49a6c44c852edee260 P 633f6da4f11c4910becf8aa67dc147c9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 07:25:35.542733 13108 tablet_bootstrap.cc:492] T dadd68ad4eb24b49a6c44c852edee260 P 2bcb44a2433144509be8441477797a48: No bootstrap required, opened a new log
I20260925 07:25:35.542783 13108 ts_tablet_manager.cc:1397] T dadd68ad4eb24b49a6c44c852edee260 P 2bcb44a2433144509be8441477797a48: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 07:25:35.543744 13107 raft_consensus.cc:348] T dadd68ad4eb24b49a6c44c852edee260 P 470984d5f90d4994b15fdb6d0d13e6c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "633f6da4f11c4910becf8aa67dc147c9" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41917 } } peers { permanent_uuid: "470984d5f90d4994b15fdb6d0d13e6c6" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 44005 } } peers { permanent_uuid: "2bcb44a2433144509be8441477797a48" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 32935 } }
I20260925 07:25:35.543841 13107 raft_consensus.cc:374] T dadd68ad4eb24b49a6c44c852edee260 P 470984d5f90d4994b15fdb6d0d13e6c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:25:35.543859 13107 raft_consensus.cc:729] T dadd68ad4eb24b49a6c44c852edee260 P 470984d5f90d4994b15fdb6d0d13e6c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 470984d5f90d4994b15fdb6d0d13e6c6, State: Initialized, Role: FOLLOWER
I20260925 07:25:35.543965 13107 consensus_queue.cc:260] T dadd68ad4eb24b49a6c44c852edee260 P 470984d5f90d4994b15fdb6d0d13e6c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "633f6da4f11c4910becf8aa67dc147c9" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41917 } } peers { permanent_uuid: "470984d5f90d4994b15fdb6d0d13e6c6" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 44005 } } peers { permanent_uuid: "2bcb44a2433144509be8441477797a48" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 32935 } }
I20260925 07:25:35.544205 13107 ts_tablet_manager.cc:1428] T dadd68ad4eb24b49a6c44c852edee260 P 470984d5f90d4994b15fdb6d0d13e6c6: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260925 07:25:35.544286 13082 heartbeater.cc:503] Master 127.5.158.126:37781 was elected leader, sending a full tablet report...
I20260925 07:25:35.544713 13109 raft_consensus.cc:348] T dadd68ad4eb24b49a6c44c852edee260 P 633f6da4f11c4910becf8aa67dc147c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "633f6da4f11c4910becf8aa67dc147c9" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41917 } } peers { permanent_uuid: "470984d5f90d4994b15fdb6d0d13e6c6" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 44005 } } peers { permanent_uuid: "2bcb44a2433144509be8441477797a48" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 32935 } }
I20260925 07:25:35.544842 13109 raft_consensus.cc:374] T dadd68ad4eb24b49a6c44c852edee260 P 633f6da4f11c4910becf8aa67dc147c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:25:35.544874 13109 raft_consensus.cc:729] T dadd68ad4eb24b49a6c44c852edee260 P 633f6da4f11c4910becf8aa67dc147c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 633f6da4f11c4910becf8aa67dc147c9, State: Initialized, Role: FOLLOWER
I20260925 07:25:35.544981 13109 consensus_queue.cc:260] T dadd68ad4eb24b49a6c44c852edee260 P 633f6da4f11c4910becf8aa67dc147c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "633f6da4f11c4910becf8aa67dc147c9" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41917 } } peers { permanent_uuid: "470984d5f90d4994b15fdb6d0d13e6c6" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 44005 } } peers { permanent_uuid: "2bcb44a2433144509be8441477797a48" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 32935 } }
I20260925 07:25:35.545207 13109 ts_tablet_manager.cc:1428] T dadd68ad4eb24b49a6c44c852edee260 P 633f6da4f11c4910becf8aa67dc147c9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 07:25:35.545120 13108 raft_consensus.cc:348] T dadd68ad4eb24b49a6c44c852edee260 P 2bcb44a2433144509be8441477797a48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "633f6da4f11c4910becf8aa67dc147c9" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41917 } } peers { permanent_uuid: "470984d5f90d4994b15fdb6d0d13e6c6" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 44005 } } peers { permanent_uuid: "2bcb44a2433144509be8441477797a48" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 32935 } }
I20260925 07:25:35.545241 13108 raft_consensus.cc:374] T dadd68ad4eb24b49a6c44c852edee260 P 2bcb44a2433144509be8441477797a48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:25:35.545269 13108 raft_consensus.cc:729] T dadd68ad4eb24b49a6c44c852edee260 P 2bcb44a2433144509be8441477797a48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bcb44a2433144509be8441477797a48, State: Initialized, Role: FOLLOWER
I20260925 07:25:35.545373 13108 consensus_queue.cc:260] T dadd68ad4eb24b49a6c44c852edee260 P 2bcb44a2433144509be8441477797a48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "633f6da4f11c4910becf8aa67dc147c9" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41917 } } peers { permanent_uuid: "470984d5f90d4994b15fdb6d0d13e6c6" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 44005 } } peers { permanent_uuid: "2bcb44a2433144509be8441477797a48" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 32935 } }
I20260925 07:25:35.545568 13108 ts_tablet_manager.cc:1428] T dadd68ad4eb24b49a6c44c852edee260 P 2bcb44a2433144509be8441477797a48: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 07:25:35.545725 12946 heartbeater.cc:503] Master 127.5.158.126:37781 was elected leader, sending a full tablet report...
I20260925 07:25:35.545936 12810 heartbeater.cc:503] Master 127.5.158.126:37781 was elected leader, sending a full tablet report...
I20260925 07:25:35.600699 13114 raft_consensus.cc:482] T dadd68ad4eb24b49a6c44c852edee260 P 633f6da4f11c4910becf8aa67dc147c9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 07:25:35.600766 13114 raft_consensus.cc:504] T dadd68ad4eb24b49a6c44c852edee260 P 633f6da4f11c4910becf8aa67dc147c9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "633f6da4f11c4910becf8aa67dc147c9" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41917 } } peers { permanent_uuid: "470984d5f90d4994b15fdb6d0d13e6c6" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 44005 } } peers { permanent_uuid: "2bcb44a2433144509be8441477797a48" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 32935 } }
I20260925 07:25:35.601052 13114 leader_election.cc:288] T dadd68ad4eb24b49a6c44c852edee260 P 633f6da4f11c4910becf8aa67dc147c9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 470984d5f90d4994b15fdb6d0d13e6c6 (127.5.158.67:44005), 2bcb44a2433144509be8441477797a48 (127.5.158.66:32935)
I20260925 07:25:35.603464 12813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:35.601224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.67:44005 (local address 127.5.158.65:38519)
0925 07:25:35.601276 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:35.601289 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:35.601337 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:35.601452 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:35.601453 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:35.601454 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:25:35.601515 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:35.601516 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.602142 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.602143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:35.602787 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:35.602789 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.603240 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.603241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:35.603305 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:35.603308 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 07:25:35.603370 (+ 62us) client_negotiation.cc:240] Negotiation successful
0925 07:25:35.603404 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":19}
I20260925 07:25:35.603459 13103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:35.601299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:38519 (local address 127.5.158.67:44005)
0925 07:25:35.601340 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:35.601342 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:35.601349 (+ 7us) server_negotiation.cc:407] Connection header received
0925 07:25:35.601395 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:35.601396 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:35.601404 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:35.601451 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:25:35.601575 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.602064 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.602862 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.603160 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.603240 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:35.603247 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:35.603377 (+ 130us) server_negotiation.cc:299] Negotiation successful
0925 07:25:35.603406 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 07:25:35.603559 13106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:35.601373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:41681 (local address 127.5.158.66:32935)
0925 07:25:35.601408 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:35.601410 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:35.601416 (+ 6us) server_negotiation.cc:407] Connection header received
0925 07:25:35.601554 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:35.601554 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:35.601561 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:35.601618 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:25:35.601748 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.602175 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.602944 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.603253 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.603324 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:35.603330 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:35.603476 (+ 146us) server_negotiation.cc:299] Negotiation successful
0925 07:25:35.603507 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 07:25:35.603570 12814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:35.601339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.66:32935 (local address 127.5.158.65:41681)
0925 07:25:35.601381 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:35.601390 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:35.601443 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:35.601621 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:35.601622 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:35.601623 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:25:35.601677 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:35.601679 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.602246 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.602247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:35.602866 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:35.602868 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.603322 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.603323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:35.603402 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:35.603404 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 07:25:35.603468 (+ 64us) client_negotiation.cc:240] Negotiation successful
0925 07:25:35.603501 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260925 07:25:35.603766 13035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dadd68ad4eb24b49a6c44c852edee260" candidate_uuid: "633f6da4f11c4910becf8aa67dc147c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "470984d5f90d4994b15fdb6d0d13e6c6" is_pre_election: true
I20260925 07:25:35.603809 12899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dadd68ad4eb24b49a6c44c852edee260" candidate_uuid: "633f6da4f11c4910becf8aa67dc147c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2bcb44a2433144509be8441477797a48" is_pre_election: true
I20260925 07:25:35.603888 13035 raft_consensus.cc:2454] T dadd68ad4eb24b49a6c44c852edee260 P 470984d5f90d4994b15fdb6d0d13e6c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 633f6da4f11c4910becf8aa67dc147c9 in term 0.
I20260925 07:25:35.603914 12899 raft_consensus.cc:2454] T dadd68ad4eb24b49a6c44c852edee260 P 2bcb44a2433144509be8441477797a48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 633f6da4f11c4910becf8aa67dc147c9 in term 0.
I20260925 07:25:35.604089 12697 leader_election.cc:302] T dadd68ad4eb24b49a6c44c852edee260 P 633f6da4f11c4910becf8aa67dc147c9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 470984d5f90d4994b15fdb6d0d13e6c6, 633f6da4f11c4910becf8aa67dc147c9; no voters:
I20260925 07:25:35.604200 13114 raft_consensus.cc:2781] T dadd68ad4eb24b49a6c44c852edee260 P 633f6da4f11c4910becf8aa67dc147c9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 07:25:35.604259 13114 raft_consensus.cc:482] T dadd68ad4eb24b49a6c44c852edee260 P 633f6da4f11c4910becf8aa67dc147c9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 07:25:35.604277 13114 raft_consensus.cc:3037] T dadd68ad4eb24b49a6c44c852edee260 P 633f6da4f11c4910becf8aa67dc147c9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:25:35.604787 13114 raft_consensus.cc:504] T dadd68ad4eb24b49a6c44c852edee260 P 633f6da4f11c4910becf8aa67dc147c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "633f6da4f11c4910becf8aa67dc147c9" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41917 } } peers { permanent_uuid: "470984d5f90d4994b15fdb6d0d13e6c6" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 44005 } } peers { permanent_uuid: "2bcb44a2433144509be8441477797a48" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 32935 } }
I20260925 07:25:35.604885 13114 leader_election.cc:288] T dadd68ad4eb24b49a6c44c852edee260 P 633f6da4f11c4910becf8aa67dc147c9 [CANDIDATE]: Term 1 election: Requested vote from peers 470984d5f90d4994b15fdb6d0d13e6c6 (127.5.158.67:44005), 2bcb44a2433144509be8441477797a48 (127.5.158.66:32935)
I20260925 07:25:35.605057 13035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dadd68ad4eb24b49a6c44c852edee260" candidate_uuid: "633f6da4f11c4910becf8aa67dc147c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "470984d5f90d4994b15fdb6d0d13e6c6"
I20260925 07:25:35.605047 12899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dadd68ad4eb24b49a6c44c852edee260" candidate_uuid: "633f6da4f11c4910becf8aa67dc147c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2bcb44a2433144509be8441477797a48"
I20260925 07:25:35.605146 12899 raft_consensus.cc:3037] T dadd68ad4eb24b49a6c44c852edee260 P 2bcb44a2433144509be8441477797a48 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:25:35.605142 13035 raft_consensus.cc:3037] T dadd68ad4eb24b49a6c44c852edee260 P 470984d5f90d4994b15fdb6d0d13e6c6 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:25:35.605712 12899 raft_consensus.cc:2454] T dadd68ad4eb24b49a6c44c852edee260 P 2bcb44a2433144509be8441477797a48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 633f6da4f11c4910becf8aa67dc147c9 in term 1.
I20260925 07:25:35.605839 13035 raft_consensus.cc:2454] T dadd68ad4eb24b49a6c44c852edee260 P 470984d5f90d4994b15fdb6d0d13e6c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 633f6da4f11c4910becf8aa67dc147c9 in term 1.
I20260925 07:25:35.605861 12697 leader_election.cc:302] T dadd68ad4eb24b49a6c44c852edee260 P 633f6da4f11c4910becf8aa67dc147c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2bcb44a2433144509be8441477797a48, 633f6da4f11c4910becf8aa67dc147c9; no voters:
I20260925 07:25:35.605949 13114 raft_consensus.cc:2781] T dadd68ad4eb24b49a6c44c852edee260 P 633f6da4f11c4910becf8aa67dc147c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:25:35.606088 13114 raft_consensus.cc:686] T dadd68ad4eb24b49a6c44c852edee260 P 633f6da4f11c4910becf8aa67dc147c9 [term 1 LEADER]: Becoming Leader. State: Replica: 633f6da4f11c4910becf8aa67dc147c9, State: Running, Role: LEADER
I20260925 07:25:35.606182 13114 consensus_queue.cc:237] T dadd68ad4eb24b49a6c44c852edee260 P 633f6da4f11c4910becf8aa67dc147c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "633f6da4f11c4910becf8aa67dc147c9" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41917 } } peers { permanent_uuid: "470984d5f90d4994b15fdb6d0d13e6c6" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 44005 } } peers { permanent_uuid: "2bcb44a2433144509be8441477797a48" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 32935 } }
I20260925 07:25:35.606858 11844 catalog_manager.cc:5495] T dadd68ad4eb24b49a6c44c852edee260 P 633f6da4f11c4910becf8aa67dc147c9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 633f6da4f11c4910becf8aa67dc147c9 (127.5.158.65). New cstate: current_term: 1 leader_uuid: "633f6da4f11c4910becf8aa67dc147c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "633f6da4f11c4910becf8aa67dc147c9" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "470984d5f90d4994b15fdb6d0d13e6c6" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 44005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2bcb44a2433144509be8441477797a48" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 32935 } health_report { overall_health: UNKNOWN } } }
I20260925 07:25:35.628618 13104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:35.626408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40350 (local address 127.5.158.65:41917)
0925 07:25:35.626447 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:35.626449 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:35.626472 (+ 23us) server_negotiation.cc:407] Connection header received
0925 07:25:35.626556 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:35.626557 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:35.626574 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:35.626635 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 07:25:35.626786 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.627229 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.627973 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.628165 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.628224 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:35.628367 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 07:25:35.628443 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 07:25:35.628519 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:35.628540 (+ 21us) server_negotiation.cc:299] Negotiation successful
0925 07:25:35.628555 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 07:25:35.629997 13035 raft_consensus.cc:1261] T dadd68ad4eb24b49a6c44c852edee260 P 470984d5f90d4994b15fdb6d0d13e6c6 [term 1 FOLLOWER]: Refusing update from remote peer 633f6da4f11c4910becf8aa67dc147c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:25:35.630016 12899 raft_consensus.cc:1261] T dadd68ad4eb24b49a6c44c852edee260 P 2bcb44a2433144509be8441477797a48 [term 1 FOLLOWER]: Refusing update from remote peer 633f6da4f11c4910becf8aa67dc147c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:25:35.630311 13116 consensus_queue.cc:1035] T dadd68ad4eb24b49a6c44c852edee260 P 633f6da4f11c4910becf8aa67dc147c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "470984d5f90d4994b15fdb6d0d13e6c6" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 44005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:25:35.630411 13114 consensus_queue.cc:1035] T dadd68ad4eb24b49a6c44c852edee260 P 633f6da4f11c4910becf8aa67dc147c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bcb44a2433144509be8441477797a48" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 32935 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:25:35.632184 13119 mvcc.cc:205] Tried to move back new op lower bound from 7333155371538726912 to 7333155371443789824. Current Snapshot: MvccSnapshot[applied={T|T < 7333155371538726912}]
I20260925 07:25:35.633368 13120 mvcc.cc:205] Tried to move back new op lower bound from 7333155371538726912 to 7333155371443789824. Current Snapshot: MvccSnapshot[applied={T|T < 7333155371538726912}]
I20260925 07:25:35.635663 11842 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 07:25:35.635748 11842 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 07:25:35.635812 13096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:35.632587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38502 (local address 127.5.158.124:36829)
0925 07:25:35.632695 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:35.632697 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:35.632702 (+ 5us) server_negotiation.cc:407] Connection header received
0925 07:25:35.632720 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:35.632721 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:35.632739 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:35.632787 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:25:35.632943 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.633558 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.634896 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.635465 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.635534 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:35.635596 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:35.635708 (+ 112us) server_negotiation.cc:299] Negotiation successful
0925 07:25:35.635750 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260925 07:25:35.636153 13105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:35.632553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.124:36829 (local address 127.0.0.1:38502)
0925 07:25:35.632588 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:35.632605 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 07:25:35.632635 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:25:35.632791 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:25:35.632791 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:25:35.632793 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:25:35.632830 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:35.632832 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.633799 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.633800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:35.634805 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:25:35.634809 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.635536 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.635537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:25:35.635585 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:35.635588 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 07:25:35.635640 (+ 52us) client_negotiation.cc:240] Negotiation successful
0925 07:25:35.635678 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260925 07:25:35.636479 12638 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 52179093193647ce9b274a063c18597b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.10] Dedup: 1.7->[1.8-1.10]
I20260925 07:25:35.636865 11842 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ea66681bd57f42b0a9db2845ec74d84d: Sending DeleteTablet for 3 replicas of tablet dadd68ad4eb24b49a6c44c852edee260
I20260925 07:25:35.637234 12879 tablet_service.cc:1514] Processing DeleteTablet for tablet dadd68ad4eb24b49a6c44c852edee260 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:25:35 UTC) from {username='oryx', principal='oryx/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:38282
I20260925 07:25:35.637287 12668 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52179093193647ce9b274a063c18597b [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea66681bd57f42b0a9db2845ec74d84d. Latest consensus state: current_term: 1 leader_uuid: "ea66681bd57f42b0a9db2845ec74d84d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ea66681bd57f42b0a9db2845ec74d84d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37781 } } peers { permanent_uuid: "87d6c9fb88664a3aa4245a34c108f564" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 44439 } attrs { promote: false } } peers { permanent_uuid: "52179093193647ce9b274a063c18597b" member_type: NON_VOTER last_known_addr { host: "127.5.158.124" port: 36829 } attrs { promote: true } } }
I20260925 07:25:35.637377 12668 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52179093193647ce9b274a063c18597b [sys.catalog]: This master's current role is: LEARNER
I20260925 07:25:35.637495 12743 tablet_service.cc:1514] Processing DeleteTablet for tablet dadd68ad4eb24b49a6c44c852edee260 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:25:35 UTC) from {username='oryx', principal='oryx/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:40342
I20260925 07:25:35.637725 13134 tablet_replica.cc:330] T dadd68ad4eb24b49a6c44c852edee260 P 633f6da4f11c4910becf8aa67dc147c9: stopping tablet replica
I20260925 07:25:35.637840 13134 raft_consensus.cc:2229] T dadd68ad4eb24b49a6c44c852edee260 P 633f6da4f11c4910becf8aa67dc147c9 [term 1 LEADER]: Raft consensus shutting down.
I20260925 07:25:35.637962 13015 tablet_service.cc:1514] Processing DeleteTablet for tablet dadd68ad4eb24b49a6c44c852edee260 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:25:35 UTC) from {username='oryx', principal='oryx/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:33592
I20260925 07:25:35.638242 13134 raft_consensus.cc:2258] T dadd68ad4eb24b49a6c44c852edee260 P 633f6da4f11c4910becf8aa67dc147c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:25:35.638597 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 11812
I20260925 07:25:35.638629 13134 ts_tablet_manager.cc:1905] T dadd68ad4eb24b49a6c44c852edee260 P 633f6da4f11c4910becf8aa67dc147c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:25:35.640043 13134 ts_tablet_manager.cc:1918] T dadd68ad4eb24b49a6c44c852edee260 P 633f6da4f11c4910becf8aa67dc147c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 07:25:35.640127 13134 log.cc:1196] T dadd68ad4eb24b49a6c44c852edee260 P 633f6da4f11c4910becf8aa67dc147c9: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-0/wal/wals/dadd68ad4eb24b49a6c44c852edee260
I20260925 07:25:35.640337 13134 ts_tablet_manager.cc:1939] T dadd68ad4eb24b49a6c44c852edee260 P 633f6da4f11c4910becf8aa67dc147c9: Deleting consensus metadata
I20260925 07:25:35.641203 13135 tablet_replica.cc:330] T dadd68ad4eb24b49a6c44c852edee260 P 470984d5f90d4994b15fdb6d0d13e6c6: stopping tablet replica
I20260925 07:25:35.641294 13135 raft_consensus.cc:2229] T dadd68ad4eb24b49a6c44c852edee260 P 470984d5f90d4994b15fdb6d0d13e6c6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 07:25:35.641377 13135 raft_consensus.cc:2258] T dadd68ad4eb24b49a6c44c852edee260 P 470984d5f90d4994b15fdb6d0d13e6c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:25:35.641506 13132 tablet_replica.cc:330] T dadd68ad4eb24b49a6c44c852edee260 P 2bcb44a2433144509be8441477797a48: stopping tablet replica
I20260925 07:25:35.641587 13132 raft_consensus.cc:2229] T dadd68ad4eb24b49a6c44c852edee260 P 2bcb44a2433144509be8441477797a48 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 07:25:35.641682 13132 raft_consensus.cc:2258] T dadd68ad4eb24b49a6c44c852edee260 P 2bcb44a2433144509be8441477797a48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:25:35.641873 13135 ts_tablet_manager.cc:1905] T dadd68ad4eb24b49a6c44c852edee260 P 470984d5f90d4994b15fdb6d0d13e6c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:25:35.643026 13132 ts_tablet_manager.cc:1905] T dadd68ad4eb24b49a6c44c852edee260 P 2bcb44a2433144509be8441477797a48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:25:35.644047 13132 ts_tablet_manager.cc:1918] T dadd68ad4eb24b49a6c44c852edee260 P 2bcb44a2433144509be8441477797a48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 07:25:35.644127 13132 log.cc:1196] T dadd68ad4eb24b49a6c44c852edee260 P 2bcb44a2433144509be8441477797a48: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-1/wal/wals/dadd68ad4eb24b49a6c44c852edee260
I20260925 07:25:35.644362 13132 ts_tablet_manager.cc:1939] T dadd68ad4eb24b49a6c44c852edee260 P 2bcb44a2433144509be8441477797a48: Deleting consensus metadata
I20260925 07:25:35.645188 13135 ts_tablet_manager.cc:1918] T dadd68ad4eb24b49a6c44c852edee260 P 470984d5f90d4994b15fdb6d0d13e6c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 07:25:35.645252 13135 log.cc:1196] T dadd68ad4eb24b49a6c44c852edee260 P 470984d5f90d4994b15fdb6d0d13e6c6: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0/minicluster-data/ts-2/wal/wals/dadd68ad4eb24b49a6c44c852edee260
W20260925 07:25:35.645423 12834 connection.cc:535] server connection from 127.0.0.1:38282 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 07:25:35.645490 13135 ts_tablet_manager.cc:1939] T dadd68ad4eb24b49a6c44c852edee260 P 470984d5f90d4994b15fdb6d0d13e6c6: Deleting consensus metadata
W20260925 07:25:35.645551 12698 connection.cc:535] server connection from 127.0.0.1:40342 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 07:25:35.645761 12970 connection.cc:441] server connection from 127.0.0.1:33592 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33592 (request call id 1) could send its response
W20260925 07:25:35.646255 12615 connection.cc:441] server connection from 127.0.0.1:38502 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38502 (request call id 0) could send its response
I20260925 07:25:35.653682 12494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:35.646846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42798 (local address 127.5.158.125:44439)
0925 07:25:35.646890 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:35.646892 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:35.647069 (+ 177us) server_negotiation.cc:407] Connection header received
0925 07:25:35.647279 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:35.647280 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:35.647291 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:35.647342 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:35.647878 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.648561 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.649495 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.649742 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.649802 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:35.651021 (+ 1219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:35.651023 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:35.651023 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:35.651030 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:35.652534 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:35.652965 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:35.652967 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:35.652967 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:35.653024 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:35.653215 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:35.653216 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:35.653216 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:35.653318 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:35.653407 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:35.653527 (+ 120us) server_negotiation.cc:299] Negotiation successful
0925 07:25:35.653600 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 07:25:35.654064 13096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:35.647054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38514 (local address 127.5.158.124:36829)
0925 07:25:35.647087 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0925 07:25:35.647089 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:25:35.647175 (+ 86us) server_negotiation.cc:407] Connection header received
0925 07:25:35.647232 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:25:35.647233 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:25:35.647242 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:25:35.647290 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:25:35.648017 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.648635 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.649290 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:25:35.649481 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:25:35.649529 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:25:35.650978 (+ 1449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:25:35.650979 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:25:35.650979 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:25:35.650986 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:25:35.652519 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:35.653109 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:35.653110 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:35.653111 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:35.653160 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:25:35.653564 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:25:35.653565 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:25:35.653565 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:25:35.653693 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:25:35.653753 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 07:25:35.653935 (+ 182us) server_negotiation.cc:299] Negotiation successful
0925 07:25:35.654003 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 07:25:36.634425 13143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:36.634039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.66:55683)
0925 07:25:36.634152 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:36.634369 (+ 217us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":59,"threads_started":1}
W20260925 07:25:36.634657 12946 heartbeater.cc:644] Failed to heartbeat to 127.5.158.126:37781 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
I20260925 07:25:36.635035 13144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:36.634831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.67:47417)
0925 07:25:36.634920 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:36.634995 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78,"thread_start_us":44,"threads_started":1}
W20260925 07:25:36.635222 13082 heartbeater.cc:644] Failed to heartbeat to 127.5.158.126:37781 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
I20260925 07:25:36.636178 13145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:36.635991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.65:38987)
0925 07:25:36.636088 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:36.636142 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82,"thread_start_us":43,"threads_started":1}
W20260925 07:25:36.636397 12810 heartbeater.cc:644] Failed to heartbeat to 127.5.158.126:37781 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
I20260925 07:25:37.137465 13146 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ea66681bd57f42b0a9db2845ec74d84d)
I20260925 07:25:37.137629 13146 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ea66681bd57f42b0a9db2845ec74d84d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37781 } } peers { permanent_uuid: "87d6c9fb88664a3aa4245a34c108f564" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 44439 } attrs { promote: false } } peers { permanent_uuid: "52179093193647ce9b274a063c18597b" member_type: NON_VOTER last_known_addr { host: "127.5.158.124" port: 36829 } attrs { promote: true } }
I20260925 07:25:37.137849 13146 leader_election.cc:288] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ea66681bd57f42b0a9db2845ec74d84d (127.5.158.126:37781)
I20260925 07:25:37.138150 13147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:37.137957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.0.0.1:59920)
0925 07:25:37.138068 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:37.138110 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":60,"threads_started":1}
W20260925 07:25:37.138273 12427 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
I20260925 07:25:37.138605 13147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:37.138512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.0.0.1:59934)
0925 07:25:37.138558 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:37.138586 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260925 07:25:37.138685 12427 leader_election.cc:334] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ea66681bd57f42b0a9db2845ec74d84d (127.5.158.126:37781): Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
I20260925 07:25:37.138751 12427 leader_election.cc:302] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 87d6c9fb88664a3aa4245a34c108f564; no voters: ea66681bd57f42b0a9db2845ec74d84d
I20260925 07:25:37.138845 13146 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 07:25:37.635311 13149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:37.635039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.66:52303)
0925 07:25:37.635195 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:37.635239 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":79,"threads_started":1}
I20260925 07:25:37.635766 13150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:37.635535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.67:57811)
0925 07:25:37.635656 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:37.635700 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":57,"threads_started":1}
I20260925 07:25:37.636832 13151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:37.636659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.65:36651)
0925 07:25:37.636767 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:37.636800 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":57,"threads_started":1}
I20260925 07:25:38.635964 13152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:38.635768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.66:41851)
0925 07:25:38.635882 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:38.635914 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":74,"threads_started":1}
I20260925 07:25:38.636343 13153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:38.636151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.67:51189)
0925 07:25:38.636259 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:38.636297 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":56,"threads_started":1}
I20260925 07:25:38.637387 13154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:38.637219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.65:41895)
0925 07:25:38.637325 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:38.637356 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":58,"threads_started":1}
I20260925 07:25:39.233755 13155 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ea66681bd57f42b0a9db2845ec74d84d)
I20260925 07:25:39.233857 13155 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ea66681bd57f42b0a9db2845ec74d84d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37781 } } peers { permanent_uuid: "87d6c9fb88664a3aa4245a34c108f564" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 44439 } attrs { promote: false } } peers { permanent_uuid: "52179093193647ce9b274a063c18597b" member_type: NON_VOTER last_known_addr { host: "127.5.158.124" port: 36829 } attrs { promote: true } }
I20260925 07:25:39.234025 13155 leader_election.cc:288] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ea66681bd57f42b0a9db2845ec74d84d (127.5.158.126:37781)
I20260925 07:25:39.234336 13156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:39.234176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.0.0.1:53926)
0925 07:25:39.234273 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:39.234301 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77,"thread_start_us":53,"threads_started":1}
I20260925 07:25:39.234740 13156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:39.234645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.0.0.1:53942)
0925 07:25:39.234699 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:39.234723 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260925 07:25:39.234813 12427 leader_election.cc:334] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ea66681bd57f42b0a9db2845ec74d84d (127.5.158.126:37781): Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
I20260925 07:25:39.234890 12427 leader_election.cc:302] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 87d6c9fb88664a3aa4245a34c108f564; no voters: ea66681bd57f42b0a9db2845ec74d84d
I20260925 07:25:39.234978 13155 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 07:25:39.636840 13159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:39.636659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.67:52533)
0925 07:25:39.636769 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:39.636793 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":47,"threads_started":1}
I20260925 07:25:39.636840 13158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:39.636596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.66:59415)
0925 07:25:39.636750 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:39.636790 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":87,"threads_started":1}
I20260925 07:25:39.637915 13160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:39.637712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.65:53601)
0925 07:25:39.637826 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:39.637862 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":55,"threads_started":1}
I20260925 07:25:40.637501 13161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:40.637274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.66:33313)
0925 07:25:40.637401 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:40.637442 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":66,"threads_started":1}
I20260925 07:25:40.637507 13162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:40.637296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.67:39383)
0925 07:25:40.637401 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:40.637456 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":54,"threads_started":1}
I20260925 07:25:40.638414 13163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:40.638267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.65:44491)
0925 07:25:40.638355 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:40.638385 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75,"thread_start_us":45,"threads_started":1}
I20260925 07:25:40.973022 13164 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ea66681bd57f42b0a9db2845ec74d84d)
I20260925 07:25:40.973135 13164 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ea66681bd57f42b0a9db2845ec74d84d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37781 } } peers { permanent_uuid: "87d6c9fb88664a3aa4245a34c108f564" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 44439 } attrs { promote: false } } peers { permanent_uuid: "52179093193647ce9b274a063c18597b" member_type: NON_VOTER last_known_addr { host: "127.5.158.124" port: 36829 } attrs { promote: true } }
I20260925 07:25:40.973322 13164 leader_election.cc:288] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ea66681bd57f42b0a9db2845ec74d84d (127.5.158.126:37781)
I20260925 07:25:40.973655 13165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:40.973476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.0.0.1:54132)
0925 07:25:40.973576 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:40.973617 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":48,"threads_started":1}
I20260925 07:25:40.974056 13165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:40.973964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.0.0.1:54148)
0925 07:25:40.974010 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:40.974037 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260925 07:25:40.974133 12427 leader_election.cc:334] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ea66681bd57f42b0a9db2845ec74d84d (127.5.158.126:37781): Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
I20260925 07:25:40.974189 12427 leader_election.cc:302] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 87d6c9fb88664a3aa4245a34c108f564; no voters: ea66681bd57f42b0a9db2845ec74d84d
I20260925 07:25:40.974342 13164 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 07:25:41.638121 13168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:41.637911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.66:48341)
0925 07:25:41.638039 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:41.638072 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":80,"threads_started":1}
I20260925 07:25:41.638129 13167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:41.637914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.67:46465)
0925 07:25:41.638038 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:41.638071 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":58,"threads_started":1}
I20260925 07:25:41.638875 13169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:41.638737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.65:57119)
0925 07:25:41.638819 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:41.638842 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70,"thread_start_us":40,"threads_started":1}
I20260925 07:25:42.638777 13171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:42.638527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.66:39865)
0925 07:25:42.638686 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:42.638727 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":88,"threads_started":1}
I20260925 07:25:42.638777 13170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:42.638515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.67:51157)
0925 07:25:42.638690 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:42.638727 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":79,"threads_started":1}
I20260925 07:25:42.639410 13172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:42.639248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.65:56993)
0925 07:25:42.639345 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:42.639377 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81,"thread_start_us":43,"threads_started":1}
I20260925 07:25:43.156607 13173 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ea66681bd57f42b0a9db2845ec74d84d)
I20260925 07:25:43.156708 13173 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ea66681bd57f42b0a9db2845ec74d84d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37781 } } peers { permanent_uuid: "87d6c9fb88664a3aa4245a34c108f564" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 44439 } attrs { promote: false } } peers { permanent_uuid: "52179093193647ce9b274a063c18597b" member_type: NON_VOTER last_known_addr { host: "127.5.158.124" port: 36829 } attrs { promote: true } }
I20260925 07:25:43.156914 13173 leader_election.cc:288] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ea66681bd57f42b0a9db2845ec74d84d (127.5.158.126:37781)
I20260925 07:25:43.157254 13174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:43.157062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.0.0.1:54376)
0925 07:25:43.157190 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:43.157222 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":78,"threads_started":1}
W20260925 07:25:43.157336 12427 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111) [suppressed 2 similar messages]
I20260925 07:25:43.157684 13174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:43.157584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.0.0.1:54382)
0925 07:25:43.157634 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:43.157657 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260925 07:25:43.157766 12427 leader_election.cc:334] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ea66681bd57f42b0a9db2845ec74d84d (127.5.158.126:37781): Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
I20260925 07:25:43.157796 12427 leader_election.cc:302] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 87d6c9fb88664a3aa4245a34c108f564; no voters: ea66681bd57f42b0a9db2845ec74d84d
I20260925 07:25:43.157862 13173 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 07:25:43.639358 13177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:43.639161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.66:55169)
0925 07:25:43.639287 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:43.639321 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":74,"threads_started":1}
I20260925 07:25:43.639358 13176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:43.639168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.67:60587)
0925 07:25:43.639290 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:43.639317 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":60,"threads_started":1}
I20260925 07:25:43.639891 13178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:43.639737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.65:40255)
0925 07:25:43.639834 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:43.639860 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":48,"threads_started":1}
I20260925 07:25:44.639919 13179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:44.639714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.67:59749)
0925 07:25:44.639835 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:44.639878 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":57,"threads_started":1}
I20260925 07:25:44.639936 13180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:44.639714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.66:41291)
0925 07:25:44.639859 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:44.639899 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":83,"threads_started":1}
I20260925 07:25:44.640342 13181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:44.640195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.65:59303)
0925 07:25:44.640284 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:44.640313 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76,"thread_start_us":41,"threads_started":1}
I20260925 07:25:45.640514 13182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:45.640323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.66:38685)
0925 07:25:45.640436 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:45.640471 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":62,"threads_started":1}
I20260925 07:25:45.640666 13183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:45.640415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.67:57245)
0925 07:25:45.640563 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:45.640602 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":70,"threads_started":1}
I20260925 07:25:45.640856 13184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:45.640697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.65:52517)
0925 07:25:45.640794 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:45.640827 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":45,"threads_started":1}
I20260925 07:25:46.641137 13185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:46.640906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.66:43597)
0925 07:25:46.641024 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:46.641063 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":60,"threads_started":1}
I20260925 07:25:46.641227 13186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:46.641035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.67:38075)
0925 07:25:46.641150 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:46.641182 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":70,"threads_started":1}
I20260925 07:25:46.641398 13187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:46.641195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.65:50107)
0925 07:25:46.641330 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:46.641371 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":44,"threads_started":1}
I20260925 07:25:47.641824 13188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:47.641536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.66:34637)
0925 07:25:47.641728 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:47.641770 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":73,"threads_started":1}
I20260925 07:25:47.641855 13189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:47.641592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.67:53515)
0925 07:25:47.641777 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:47.641817 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":68,"threads_started":1}
I20260925 07:25:47.641968 13190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:47.641810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.65:52747)
0925 07:25:47.641906 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:47.641933 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":44,"threads_started":1}
I20260925 07:25:48.642472 13192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:48.642209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.67:52579)
0925 07:25:48.642368 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:48.642412 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":68,"threads_started":1}
I20260925 07:25:48.642668 13191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:48.642209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.66:42309)
0925 07:25:48.642595 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:48.642631 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":66,"threads_started":1}
I20260925 07:25:48.642693 13193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:48.642471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.65:60691)
0925 07:25:48.642629 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:48.642655 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":66,"threads_started":1}
I20260925 07:25:48.769172 13194 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ea66681bd57f42b0a9db2845ec74d84d)
I20260925 07:25:48.769253 13194 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ea66681bd57f42b0a9db2845ec74d84d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37781 } } peers { permanent_uuid: "87d6c9fb88664a3aa4245a34c108f564" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 44439 } attrs { promote: false } } peers { permanent_uuid: "52179093193647ce9b274a063c18597b" member_type: NON_VOTER last_known_addr { host: "127.5.158.124" port: 36829 } attrs { promote: true } }
I20260925 07:25:48.769392 13194 leader_election.cc:288] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ea66681bd57f42b0a9db2845ec74d84d (127.5.158.126:37781)
I20260925 07:25:48.769675 13195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:48.769507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.0.0.1:44424)
0925 07:25:48.769604 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:48.769640 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82,"thread_start_us":47,"threads_started":1}
W20260925 07:25:48.769765 12427 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
I20260925 07:25:48.770107 13195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:48.770025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.0.0.1:44430)
0925 07:25:48.770068 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:48.770091 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260925 07:25:48.770174 12427 leader_election.cc:334] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ea66681bd57f42b0a9db2845ec74d84d (127.5.158.126:37781): Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
I20260925 07:25:48.770243 12427 leader_election.cc:302] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 87d6c9fb88664a3aa4245a34c108f564; no voters: ea66681bd57f42b0a9db2845ec74d84d
I20260925 07:25:48.770326 13194 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 07:25:49.643214 13197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:49.642902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.67:45567)
0925 07:25:49.643122 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:49.643159 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":69,"threads_started":1}
I20260925 07:25:49.643476 13199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:49.643121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.66:51513)
0925 07:25:49.643275 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:49.643306 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":54,"threads_started":1}
I20260925 07:25:49.643456 13198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:49.643096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.65:44415)
0925 07:25:49.643389 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:49.643421 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":44,"threads_started":1}
I20260925 07:25:50.643983 13200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:50.643776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.67:36263)
0925 07:25:50.643889 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:50.643929 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":58,"threads_started":1}
I20260925 07:25:50.644104 13201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:50.643833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.65:38727)
0925 07:25:50.644024 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:50.644058 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":38,"threads_started":1}
I20260925 07:25:50.644188 13202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:50.643886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.66:40223)
0925 07:25:50.644070 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:50.644097 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":49,"threads_started":1}
I20260925 07:25:51.644788 13203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:51.644446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.67:57623)
0925 07:25:51.644672 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:51.644713 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":63,"threads_started":1}
I20260925 07:25:51.645066 13204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:51.644571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.65:34975)
0925 07:25:51.644982 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:51.645022 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":43,"threads_started":1}
I20260925 07:25:51.645135 13205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:51.644724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.66:60065)
0925 07:25:51.645032 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:51.645066 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":66,"threads_started":1}
I20260925 07:25:52.645664 13206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:52.645419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.67:46207)
0925 07:25:52.645569 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:52.645606 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":62,"threads_started":1}
I20260925 07:25:52.645819 13208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:52.645600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.65:48245)
0925 07:25:52.645742 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:52.645777 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":67,"threads_started":1}
I20260925 07:25:52.646106 13207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:52.645532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.66:38429)
0925 07:25:52.646029 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:52.646066 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":42,"threads_started":1}
I20260925 07:25:53.646474 13209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:53.646225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.65:40153)
0925 07:25:53.646382 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:53.646422 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":69,"threads_started":1}
I20260925 07:25:53.646632 13211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:53.646477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.66:57107)
0925 07:25:53.646569 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:53.646595 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79,"thread_start_us":44,"threads_started":1}
I20260925 07:25:53.646632 13210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:53.646253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.67:57869)
0925 07:25:53.646533 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:53.646570 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":60,"threads_started":1}
I20260925 07:25:54.647508 13213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:54.647057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.67:38667)
0925 07:25:54.647407 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:54.647452 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":65,"threads_started":1}
I20260925 07:25:54.647547 13214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:54.647274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.66:34937)
0925 07:25:54.647423 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:54.647468 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":49,"threads_started":1}
I20260925 07:25:54.647670 13212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:54.647057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.65:38861)
0925 07:25:54.647595 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:54.647629 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":68,"threads_started":1}
I20260925 07:25:55.648306 13215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:55.648006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.66:45989)
0925 07:25:55.648208 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:55.648251 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":68,"threads_started":1}
I20260925 07:25:55.648331 13217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:55.648085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.67:57691)
0925 07:25:55.648268 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:55.648297 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":43,"threads_started":1}
I20260925 07:25:55.648326 13216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:55.648011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.65:55767)
0925 07:25:55.648243 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:55.648282 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":65,"threads_started":1}
I20260925 07:25:56.649040 13218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:56.648790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.66:57269)
0925 07:25:56.648950 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:56.648989 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":60,"threads_started":1}
I20260925 07:25:56.649040 13219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:56.648790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.65:52213)
0925 07:25:56.648968 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:56.648993 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":66,"threads_started":1}
I20260925 07:25:56.649075 13220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:56.648865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.67:50883)
0925 07:25:56.648994 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:56.649027 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":54,"threads_started":1}
I20260925 07:25:57.649735 13222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:57.649531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.67:35897)
0925 07:25:57.649641 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:57.649679 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":53,"threads_started":1}
I20260925 07:25:57.649734 13221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:57.649508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.66:41975)
0925 07:25:57.649639 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:57.649679 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":63,"threads_started":1}
I20260925 07:25:57.649783 13223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:57.649623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.65:47537)
0925 07:25:57.649718 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:57.649746 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82,"thread_start_us":47,"threads_started":1}
I20260925 07:25:58.650462 13224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:58.650185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.65:54283)
0925 07:25:58.650366 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:58.650407 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":79,"threads_started":1}
I20260925 07:25:58.650585 13226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:58.650401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.66:56399)
0925 07:25:58.650511 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:58.650542 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":43,"threads_started":1}
I20260925 07:25:58.650585 13225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:58.650298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.67:60291)
0925 07:25:58.650511 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:58.650542 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":54,"threads_started":1}
I20260925 07:25:58.873373 13227 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ea66681bd57f42b0a9db2845ec74d84d)
I20260925 07:25:58.873448 13227 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ea66681bd57f42b0a9db2845ec74d84d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37781 } } peers { permanent_uuid: "87d6c9fb88664a3aa4245a34c108f564" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 44439 } attrs { promote: false } } peers { permanent_uuid: "52179093193647ce9b274a063c18597b" member_type: NON_VOTER last_known_addr { host: "127.5.158.124" port: 36829 } attrs { promote: true } }
I20260925 07:25:58.873565 13227 leader_election.cc:288] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ea66681bd57f42b0a9db2845ec74d84d (127.5.158.126:37781)
I20260925 07:25:58.873996 13228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:58.873740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.0.0.1:46712)
0925 07:25:58.873871 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:58.873913 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":76,"threads_started":1}
W20260925 07:25:58.874073 12427 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
I20260925 07:25:58.874423 13228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:58.874332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.0.0.1:46728)
0925 07:25:58.874380 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:58.874405 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260925 07:25:58.874495 12427 leader_election.cc:334] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ea66681bd57f42b0a9db2845ec74d84d (127.5.158.126:37781): Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
I20260925 07:25:58.874563 12427 leader_election.cc:302] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 87d6c9fb88664a3aa4245a34c108f564; no voters: ea66681bd57f42b0a9db2845ec74d84d
I20260925 07:25:58.874645 13227 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 87d6c9fb88664a3aa4245a34c108f564 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 07:25:59.651312 13230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:59.651017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.67:46689)
0925 07:25:59.651170 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:59.651217 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":65,"threads_started":1}
I20260925 07:25:59.651396 13232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:59.651225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.65:34587)
0925 07:25:59.651327 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:59.651355 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":52,"threads_started":1}
I20260925 07:25:59.651521 13231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:25:59.651181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.66:49789)
0925 07:25:59.651428 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0925 07:25:59.651467 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":51,"threads_started":1}
I20260925 07:26:00.651971 13233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:00.651714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.65:43101)
0925 07:26:00.651855 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:00.651898 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":78,"threads_started":1}
I20260925 07:26:00.652048 13234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:00.651857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.66:39233)
0925 07:26:00.651973 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:00.652007 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":60,"threads_started":1}
I20260925 07:26:00.652194 13235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:00.651925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.67:53881)
0925 07:26:00.652045 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:00.652149 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":43,"threads_started":1}
I20260925 07:26:01.652702 13236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:01.652429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.65:50779)
0925 07:26:01.652585 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:01.652645 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":61,"threads_started":1}
I20260925 07:26:01.652823 13237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:01.652429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.66:55597)
0925 07:26:01.652733 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:01.652773 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":75,"threads_started":1}
I20260925 07:26:01.652906 13238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:01.652735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.67:60843)
0925 07:26:01.652838 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:01.652868 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":66,"threads_started":1}
I20260925 07:26:02.653544 13239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:02.653287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.65:33857)
0925 07:26:02.653443 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:02.653486 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":68,"threads_started":1}
I20260925 07:26:02.653638 13240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:02.653294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.66:49847)
0925 07:26:02.653555 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:02.653587 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":56,"threads_started":1}
I20260925 07:26:02.653649 13241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:02.653444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.67:51211)
0925 07:26:02.653549 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:02.653587 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":41,"threads_started":1}
I20260925 07:26:03.654213 13242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:03.653978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.66:42869)
0925 07:26:03.654089 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:03.654126 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":61,"threads_started":1}
I20260925 07:26:03.654395 13243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:03.654081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.67:51721)
0925 07:26:03.654336 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:03.654362 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":44,"threads_started":1}
I20260925 07:26:03.654383 13244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:03.654121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.65:50319)
0925 07:26:03.654292 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:03.654328 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":69,"threads_started":1}
I20260925 07:26:04.655151 13247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:04.654948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.67:42179)
0925 07:26:04.655060 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:04.655091 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":45,"threads_started":1}
I20260925 07:26:04.655105 13246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:04.654752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.66:46699)
0925 07:26:04.654975 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:04.655048 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":86,"threads_started":1}
I20260925 07:26:04.655227 13245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:04.654741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.65:34713)
0925 07:26:04.655034 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:04.655071 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":67,"threads_started":1}
I20260925 07:26:05.655723 13248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:05.655471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.66:57733)
0925 07:26:05.655604 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:05.655646 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":64,"threads_started":1}
I20260925 07:26:05.655997 13249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:05.655675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.65:54979)
0925 07:26:05.655890 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:05.655928 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":52,"threads_started":1}
I20260925 07:26:05.656026 13250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:05.655782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37781 (local address 127.5.158.67:43357)
0925 07:26:05.655931 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:05.655963 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":86,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:903: Failure
Failed
Bad status: Timed out: Could not connect to the cluster: ConnectToClusterRpc(addrs: 127.5.158.126:37781,127.5.158.125:44439,127.5.158.124:36829, num_attempts: 243) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.5.158.126:37781,127.5.158.125:44439,127.5.158.124:36829, num_attempts: 1)
I20260925 07:26:05.803097 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 12680
I20260925 07:26:05.808471 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 12816
I20260925 07:26:05.812808 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 12952
I20260925 07:26:05.816862 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 12308
I20260925 07:26:05.820986 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 12497
2026-09-25T07:26:05Z chronyd exiting
I20260925 07:26:05.836427 5753 test_util.cc:175] -----------------------------------------------
I20260925 07:26:05.836484 5753 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790321110323796-5753-0
[ FAILED ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (32772 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 07:26:05 dist-test-slave-1drq krb5kdc[13253](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 07:26:05 dist-test-slave-1drq krb5kdc[13253](info): set up 2 sockets
Sep 25 07:26:05 dist-test-slave-1drq krb5kdc[13253](info): commencing operation
krb5kdc: starting...
W20260925 07:26:06.780977 5753 mini_kdc.cc:121] Time spent starting KDC: real 0.943s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Sep 25 07:26:06 dist-test-slave-1drq krb5kdc[13253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321166, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T07:26:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T07:26:06Z Disabled control of system clock
WARNING: no policy specified for oryx/127.5.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/krb5kdc/oryx_127.5.158.126.keytab.
Entry for principal oryx/127.5.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/krb5kdc/oryx_127.5.158.126.keytab.
I20260925 07:26:06.882093 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:40461
--webserver_interface=127.5.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/krb5kdc/oryx_127.5.158.126.keytab
--principal=oryx/127.5.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:42619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.158.126:40461
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:26:06.955536 13266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:26:06.955749 13266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:26:06.955783 13266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:26:06.956923 13266 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:26:06.956992 13266 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260925 07:26:06.957017 13266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:26:06.957049 13266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:26:06.957075 13266 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:26:06.957113 13266 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:26:06.958525 13266 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:42619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.158.126:40461
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/krb5kdc/oryx_127.5.158.126.keytab
--principal=oryx/127.5.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:40461
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13266
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:26:06.958774 13266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:26:06.958971 13266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 07:26:06.963863 13266 server_base.cc:1034] running on GCE node
W20260925 07:26:06.963840 13271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:26:06.963825 13272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:26:06.963831 13274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:26:06.964231 13266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:26:06.964502 13266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:26:06.965670 13266 hybrid_clock.cc:648] HybridClock initialized: now 1790321166965657 us; error 43 us; skew 500 ppm
Sep 25 07:26:06 dist-test-slave-1drq krb5kdc[13253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321166, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:26:06.968097 13266 init.cc:377] Logged in from keytab as oryx/127.5.158.126@KRBTEST.COM (short username oryx)
I20260925 07:26:06.968468 13266 webserver.cc:460] Webserver started at http://127.5.158.126:37035/ using document root <none> and password file <none>
I20260925 07:26:06.968626 13266 fs_manager.cc:359] Metadata directory not provided
I20260925 07:26:06.968659 13266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:26:06.968734 13266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:26:06.969676 13266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/master-0/data/instance:
uuid: "f5e596c5f3284f62a2c097cde2041a96"
format_stamp: "Formatted at 2026-09-25 07:26:06 on dist-test-slave-1drq"
server_key: "2205b2a12f4f4d793cf55dff9e4d9db3"
server_key_iv: "0860a8a7af64d14a7191f62694e8dabe"
server_key_version: "encryptionkey@0"
I20260925 07:26:06.969949 13266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/master-0/wal/instance:
uuid: "f5e596c5f3284f62a2c097cde2041a96"
format_stamp: "Formatted at 2026-09-25 07:26:06 on dist-test-slave-1drq"
server_key: "2205b2a12f4f4d793cf55dff9e4d9db3"
server_key_iv: "0860a8a7af64d14a7191f62694e8dabe"
server_key_version: "encryptionkey@0"
I20260925 07:26:06.971067 13266 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 07:26:06.971645 13281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:06.971799 13266 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:26:06.971897 13266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "f5e596c5f3284f62a2c097cde2041a96"
format_stamp: "Formatted at 2026-09-25 07:26:06 on dist-test-slave-1drq"
server_key: "2205b2a12f4f4d793cf55dff9e4d9db3"
server_key_iv: "0860a8a7af64d14a7191f62694e8dabe"
server_key_version: "encryptionkey@0"
I20260925 07:26:06.971966 13266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:26:06.979497 13266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:26:06.981467 13266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:26:06.981622 13266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:26:06.985807 13266 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.126:40461
I20260925 07:26:06.985857 13343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.126:40461 every 8 connection(s)
I20260925 07:26:06.986153 13266 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:26:06.986688 13344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:26:06.988613 13344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5e596c5f3284f62a2c097cde2041a96: Bootstrap starting.
I20260925 07:26:06.989190 13344 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f5e596c5f3284f62a2c097cde2041a96: Neither blocks nor log segments found. Creating new log.
I20260925 07:26:06.989465 13344 log.cc:824] T 00000000000000000000000000000000 P f5e596c5f3284f62a2c097cde2041a96: Log is configured to *not* fsync() on all Append() calls
I20260925 07:26:06.990027 13344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5e596c5f3284f62a2c097cde2041a96: No bootstrap required, opened a new log
I20260925 07:26:06.991792 13344 raft_consensus.cc:348] T 00000000000000000000000000000000 P f5e596c5f3284f62a2c097cde2041a96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5e596c5f3284f62a2c097cde2041a96" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 40461 } }
I20260925 07:26:06.991925 13344 raft_consensus.cc:374] T 00000000000000000000000000000000 P f5e596c5f3284f62a2c097cde2041a96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:26:06.991967 13344 raft_consensus.cc:729] T 00000000000000000000000000000000 P f5e596c5f3284f62a2c097cde2041a96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5e596c5f3284f62a2c097cde2041a96, State: Initialized, Role: FOLLOWER
I20260925 07:26:06.992091 13344 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5e596c5f3284f62a2c097cde2041a96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5e596c5f3284f62a2c097cde2041a96" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 40461 } }
I20260925 07:26:06.992172 13344 raft_consensus.cc:388] T 00000000000000000000000000000000 P f5e596c5f3284f62a2c097cde2041a96 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 07:26:06.992219 13344 raft_consensus.cc:482] T 00000000000000000000000000000000 P f5e596c5f3284f62a2c097cde2041a96 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 07:26:06.992280 13344 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f5e596c5f3284f62a2c097cde2041a96 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:26:06.992738 13344 raft_consensus.cc:504] T 00000000000000000000000000000000 P f5e596c5f3284f62a2c097cde2041a96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5e596c5f3284f62a2c097cde2041a96" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 40461 } }
I20260925 07:26:06.992839 13344 leader_election.cc:302] T 00000000000000000000000000000000 P f5e596c5f3284f62a2c097cde2041a96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f5e596c5f3284f62a2c097cde2041a96; no voters:
I20260925 07:26:06.992995 13344 leader_election.cc:288] T 00000000000000000000000000000000 P f5e596c5f3284f62a2c097cde2041a96 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 07:26:06.993031 13347 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f5e596c5f3284f62a2c097cde2041a96 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:26:06.993228 13347 raft_consensus.cc:686] T 00000000000000000000000000000000 P f5e596c5f3284f62a2c097cde2041a96 [term 1 LEADER]: Becoming Leader. State: Replica: f5e596c5f3284f62a2c097cde2041a96, State: Running, Role: LEADER
I20260925 07:26:06.993239 13344 sys_catalog.cc:564] T 00000000000000000000000000000000 P f5e596c5f3284f62a2c097cde2041a96 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:26:06.993319 13347 consensus_queue.cc:237] T 00000000000000000000000000000000 P f5e596c5f3284f62a2c097cde2041a96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5e596c5f3284f62a2c097cde2041a96" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 40461 } }
I20260925 07:26:06.993901 13348 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5e596c5f3284f62a2c097cde2041a96 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f5e596c5f3284f62a2c097cde2041a96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5e596c5f3284f62a2c097cde2041a96" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 40461 } } }
I20260925 07:26:06.993957 13349 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5e596c5f3284f62a2c097cde2041a96 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5e596c5f3284f62a2c097cde2041a96. Latest consensus state: current_term: 1 leader_uuid: "f5e596c5f3284f62a2c097cde2041a96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5e596c5f3284f62a2c097cde2041a96" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 40461 } } }
I20260925 07:26:06.994017 13348 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5e596c5f3284f62a2c097cde2041a96 [sys.catalog]: This master's current role is: LEADER
I20260925 07:26:06.994047 13349 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5e596c5f3284f62a2c097cde2041a96 [sys.catalog]: This master's current role is: LEADER
I20260925 07:26:06.994227 13356 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 07:26:06.994792 13356 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 07:26:06.995882 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 13266
I20260925 07:26:06.995981 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/master-0/wal/instance
I20260925 07:26:06.996133 5753 external_mini_cluster.cc:1428] Setting key 082f988b0565675316df77d5b467b799
I20260925 07:26:06.996266 13356 catalog_manager.cc:1347] Generated new cluster ID: a86822c1c89547ad907ad08812b7243a
I20260925 07:26:06.996309 13356 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 07:26:07 dist-test-slave-1drq krb5kdc[13253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321166, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM
I20260925 07:26:07.009837 13367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:06.996519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50770 (local address 127.5.158.126:40461)
0925 07:26:07.001216 (+ 4697us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:07.001223 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:07.001248 (+ 25us) server_negotiation.cc:407] Connection header received
0925 07:26:07.001329 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:07.001335 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:07.001377 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:07.001527 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:07.002151 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.003277 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.004095 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.004469 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.004576 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:07.006684 (+ 2108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:26:07.006687 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:26:07.006688 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:26:07.006700 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:26:07.008288 (+ 1588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:26:07.008674 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:26:07.008677 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:26:07.008677 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:26:07.008758 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:26:07.008951 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:26:07.008951 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:26:07.008952 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:26:07.009259 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:26:07.009420 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0925 07:26:07.009495 (+ 75us) server_negotiation.cc:299] Negotiation successful
0925 07:26:07.009626 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4658,"thread_start_us":81,"threads_started":1}
I20260925 07:26:07.018380 13356 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 07:26:07.018836 13356 catalog_manager.cc:1504] Loading token signing keys...
I20260925 07:26:07.022140 13356 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f5e596c5f3284f62a2c097cde2041a96: Generated new TSK 0
I20260925 07:26:07.022286 13356 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.5.158.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.158.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.158.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/krb5kdc/oryx_127.5.158.65.keytab.
Entry for principal oryx/127.5.158.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/krb5kdc/oryx_127.5.158.65.keytab.
I20260925 07:26:07.047451 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.65:0
--local_ip_for_outbound_sockets=127.5.158.65
--webserver_interface=127.5.158.65
--webserver_port=0
--tserver_master_addrs=127.5.158.126:40461
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/krb5kdc/oryx_127.5.158.65.keytab
--principal=oryx/127.5.158.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:42619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:26:07.120997 13370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:26:07.121187 13370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:26:07.121212 13370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:26:07.122221 13370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:26:07.122252 13370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:26:07.122270 13370 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 07:26:07.122287 13370 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.65
I20260925 07:26:07.123550 13370 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:42619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/krb5kdc/oryx_127.5.158.65.keytab
--principal=oryx/127.5.158.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.158.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.158.126:40461
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13370
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.65
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:26:07.123720 13370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:26:07.123863 13370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:26:07.128769 13378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:26:07.128769 13375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:26:07.128793 13370 server_base.cc:1034] running on GCE node
W20260925 07:26:07.128793 13376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:26:07.129160 13370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:26:07.129382 13370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:26:07.130534 13370 hybrid_clock.cc:648] HybridClock initialized: now 1790321167130507 us; error 44 us; skew 500 ppm
Sep 25 07:26:07 dist-test-slave-1drq krb5kdc[13253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321167, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.158.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:26:07.132833 13370 init.cc:377] Logged in from keytab as oryx/127.5.158.65@KRBTEST.COM (short username oryx)
I20260925 07:26:07.133315 13370 webserver.cc:460] Webserver started at http://127.5.158.65:41267/ using document root <none> and password file <none>
I20260925 07:26:07.133529 13370 fs_manager.cc:359] Metadata directory not provided
I20260925 07:26:07.133592 13370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:26:07.133713 13370 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:26:07.134600 13370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-0/data/instance:
uuid: "1353649238cf4f7a909ccd8e1462cecd"
format_stamp: "Formatted at 2026-09-25 07:26:07 on dist-test-slave-1drq"
server_key: "ff5244b37346463b593645b09d616717"
server_key_iv: "59e8bb9eea8363d90d8e7f7090ded668"
server_key_version: "encryptionkey@0"
I20260925 07:26:07.134924 13370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-0/wal/instance:
uuid: "1353649238cf4f7a909ccd8e1462cecd"
format_stamp: "Formatted at 2026-09-25 07:26:07 on dist-test-slave-1drq"
server_key: "ff5244b37346463b593645b09d616717"
server_key_iv: "59e8bb9eea8363d90d8e7f7090ded668"
server_key_version: "encryptionkey@0"
I20260925 07:26:07.136152 13370 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 07:26:07.136849 13385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:07.137012 13370 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:26:07.137123 13370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-0/wal
uuid: "1353649238cf4f7a909ccd8e1462cecd"
format_stamp: "Formatted at 2026-09-25 07:26:07 on dist-test-slave-1drq"
server_key: "ff5244b37346463b593645b09d616717"
server_key_iv: "59e8bb9eea8363d90d8e7f7090ded668"
server_key_version: "encryptionkey@0"
I20260925 07:26:07.137212 13370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:26:07.151805 13370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:26:07.153524 13370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:26:07.153671 13370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:26:07.154111 13370 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:26:07.154170 13370 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:07.154201 13370 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:26:07.154228 13370 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 07:26:07 dist-test-slave-1drq krb5kdc[13253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321167, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.158.65@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM
I20260925 07:26:07.160473 13370 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.65:41851
I20260925 07:26:07.160573 13500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.65:41851 every 8 connection(s)
I20260925 07:26:07.160825 13370 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
I20260925 07:26:07.161253 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 13370
I20260925 07:26:07.161368 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-0/wal/instance
I20260925 07:26:07.161588 5753 external_mini_cluster.cc:1428] Setting key d5786e99596c6c11731c6f9ab74b4d3d
I20260925 07:26:07.164072 13367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:07.154554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:34789 (local address 127.5.158.126:40461)
0925 07:26:07.154617 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:07.154619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:07.154780 (+ 161us) server_negotiation.cc:407] Connection header received
0925 07:26:07.155224 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:07.155225 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:07.155239 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:07.155285 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:07.156033 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.156820 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.158455 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.158670 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.158724 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:07.160877 (+ 2153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:26:07.160878 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:26:07.160879 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:26:07.160885 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:26:07.162655 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:26:07.163066 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:26:07.163067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:26:07.163068 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:26:07.163127 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:26:07.163402 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:26:07.163403 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:26:07.163404 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:26:07.163519 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:26:07.163577 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 07:26:07.163912 (+ 335us) server_negotiation.cc:299] Negotiation successful
0925 07:26:07.163986 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 07:26:07.164285 13394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:07.154511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:40461 (local address 127.5.158.65:34789)
0925 07:26:07.154683 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:07.154717 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:07.155121 (+ 404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:07.155342 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:07.155345 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:07.155706 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:07.155952 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:07.155957 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.156938 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.156940 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:07.158349 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:07.158353 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.158745 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.158746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:07.158889 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:07.159389 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:07.159398 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:07.160791 (+ 1393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:26:07.162749 (+ 1958us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:26:07.162756 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:26:07.162758 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:26:07.162973 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:26:07.163211 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:26:07.163212 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:26:07.163212 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:26:07.163346 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:26:07.163589 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:26:07.163595 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:26:07.163771 (+ 176us) client_negotiation.cc:769] Sending connection context
0925 07:26:07.163892 (+ 121us) client_negotiation.cc:240] Negotiation successful
0925 07:26:07.164054 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":78,"threads_started":1}
I20260925 07:26:07.164691 13501 heartbeater.cc:347] Connected to a master server at 127.5.158.126:40461
I20260925 07:26:07.164774 13501 heartbeater.cc:464] Registering TS with master...
I20260925 07:26:07.164955 13501 heartbeater.cc:511] Master 127.5.158.126:40461 requested a full tablet report, sending...
I20260925 07:26:07.165465 13297 ts_manager.cc:194] Registered new tserver with Master: 1353649238cf4f7a909ccd8e1462cecd (127.5.158.65:41851)
I20260925 07:26:07.166348 13297 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.158.65@KRBTEST.COM'} at 127.5.158.65:34789
WARNING: no policy specified for oryx/127.5.158.66@KRBTEST.COM; defaulting to no policy
I20260925 07:26:07.171169 13367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:07.165749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:53095 (local address 127.5.158.126:40461)
0925 07:26:07.165785 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:07.165787 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:07.165936 (+ 149us) server_negotiation.cc:407] Connection header received
0925 07:26:07.165964 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:07.165964 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:07.165975 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:07.166035 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:07.166320 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.167125 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.167759 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.168035 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.168109 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:07.168993 (+ 884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:26:07.168994 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:26:07.168994 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:26:07.169000 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:26:07.170345 (+ 1345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:26:07.170626 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:26:07.170626 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:26:07.170627 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:26:07.170669 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:26:07.170797 (+ 128us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:26:07.170797 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:26:07.170797 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:26:07.170869 (+ 72us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:26:07.170906 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0925 07:26:07.171071 (+ 165us) server_negotiation.cc:299] Negotiation successful
0925 07:26:07.171122 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 07:26:07.171185 13508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:07.165738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:40461 (local address 127.5.158.65:53095)
0925 07:26:07.165860 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:07.165869 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:07.165924 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:07.166040 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:07.166040 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:07.166200 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:07.166251 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:07.166253 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.167203 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.167203 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:07.167681 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:07.167682 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.168128 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.168129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:07.168181 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:07.168577 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:07.168581 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:07.168911 (+ 330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:26:07.170409 (+ 1498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:26:07.170409 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:26:07.170410 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:26:07.170566 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:26:07.170720 (+ 154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:26:07.170720 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:26:07.170721 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:26:07.170764 (+ 43us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:26:07.170920 (+ 156us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:26:07.170921 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:26:07.170976 (+ 55us) client_negotiation.cc:769] Sending connection context
0925 07:26:07.171037 (+ 61us) client_negotiation.cc:240] Negotiation successful
0925 07:26:07.171132 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":68,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.158.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.158.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/krb5kdc/oryx_127.5.158.66.keytab.
Entry for principal oryx/127.5.158.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/krb5kdc/oryx_127.5.158.66.keytab.
I20260925 07:26:07.187206 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.66:0
--local_ip_for_outbound_sockets=127.5.158.66
--webserver_interface=127.5.158.66
--webserver_port=0
--tserver_master_addrs=127.5.158.126:40461
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/krb5kdc/oryx_127.5.158.66.keytab
--principal=oryx/127.5.158.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:42619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:26:07.259912 13510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:26:07.260100 13510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:26:07.260123 13510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:26:07.261217 13510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:26:07.261250 13510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:26:07.261270 13510 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 07:26:07.261286 13510 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.66
I20260925 07:26:07.262686 13510 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:42619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/krb5kdc/oryx_127.5.158.66.keytab
--principal=oryx/127.5.158.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.158.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.158.126:40461
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13510
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.66
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:26:07.262858 13510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:26:07.263007 13510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:26:07.267846 13516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:26:07.267849 13515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:26:07.267846 13510 server_base.cc:1034] running on GCE node
W20260925 07:26:07.267859 13518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:26:07.268365 13510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:26:07.268630 13510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:26:07.269793 13510 hybrid_clock.cc:648] HybridClock initialized: now 1790321167269780 us; error 24 us; skew 500 ppm
Sep 25 07:26:07 dist-test-slave-1drq krb5kdc[13253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321167, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.158.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:26:07.272136 13510 init.cc:377] Logged in from keytab as oryx/127.5.158.66@KRBTEST.COM (short username oryx)
I20260925 07:26:07.272554 13510 webserver.cc:460] Webserver started at http://127.5.158.66:38629/ using document root <none> and password file <none>
I20260925 07:26:07.272754 13510 fs_manager.cc:359] Metadata directory not provided
I20260925 07:26:07.272815 13510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:26:07.272974 13510 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:26:07.273962 13510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-1/data/instance:
uuid: "156cfffb10f04e838f2b80e19e59824a"
format_stamp: "Formatted at 2026-09-25 07:26:07 on dist-test-slave-1drq"
server_key: "03666082a5502051a1ba5f5c1bd6abf5"
server_key_iv: "f90dcfdd4ffa8b7b7d44d4bc269ca876"
server_key_version: "encryptionkey@0"
I20260925 07:26:07.274295 13510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-1/wal/instance:
uuid: "156cfffb10f04e838f2b80e19e59824a"
format_stamp: "Formatted at 2026-09-25 07:26:07 on dist-test-slave-1drq"
server_key: "03666082a5502051a1ba5f5c1bd6abf5"
server_key_iv: "f90dcfdd4ffa8b7b7d44d4bc269ca876"
server_key_version: "encryptionkey@0"
I20260925 07:26:07.275460 13510 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 07:26:07.276091 13525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:07.276247 13510 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:26:07.276331 13510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-1/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-1/wal
uuid: "156cfffb10f04e838f2b80e19e59824a"
format_stamp: "Formatted at 2026-09-25 07:26:07 on dist-test-slave-1drq"
server_key: "03666082a5502051a1ba5f5c1bd6abf5"
server_key_iv: "f90dcfdd4ffa8b7b7d44d4bc269ca876"
server_key_version: "encryptionkey@0"
I20260925 07:26:07.276407 13510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:26:07.303548 13510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:26:07.305189 13510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:26:07.305321 13510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:26:07.305768 13510 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:26:07.305826 13510 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:07.305866 13510 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:26:07.305891 13510 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 07:26:07 dist-test-slave-1drq krb5kdc[13253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321167, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.158.66@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM
I20260925 07:26:07.313122 13510 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.66:36529
I20260925 07:26:07.313194 13640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.66:36529 every 8 connection(s)
I20260925 07:26:07.313511 13510 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
I20260925 07:26:07.315589 13367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:07.306189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:41051 (local address 127.5.158.126:40461)
0925 07:26:07.306255 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:07.306257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:07.306439 (+ 182us) server_negotiation.cc:407] Connection header received
0925 07:26:07.306868 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:07.306869 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:07.306883 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:07.306966 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:07.307757 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.308552 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.310218 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.310460 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.310540 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:07.313355 (+ 2815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:26:07.313357 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:26:07.313357 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:26:07.313364 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:26:07.314645 (+ 1281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:26:07.314948 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:26:07.314948 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:26:07.314949 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:26:07.314993 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:26:07.315170 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:26:07.315171 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:26:07.315171 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:26:07.315253 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:26:07.315296 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0925 07:26:07.315483 (+ 187us) server_negotiation.cc:299] Negotiation successful
0925 07:26:07.315533 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 07:26:07.315748 13534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:07.306160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:40461 (local address 127.5.158.66:41051)
0925 07:26:07.306312 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:07.306344 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:07.306774 (+ 430us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:07.307010 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:07.307013 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:07.307372 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:07.307653 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:07.307658 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.308653 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.308654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:07.310120 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:07.310123 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.310570 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.310571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:07.310747 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:07.311271 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:07.311283 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:07.313239 (+ 1956us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:26:07.314706 (+ 1467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:26:07.314710 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:26:07.314711 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:26:07.314887 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:26:07.315042 (+ 155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:26:07.315043 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:26:07.315043 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:26:07.315122 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:26:07.315301 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:26:07.315304 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:26:07.315391 (+ 87us) client_negotiation.cc:769] Sending connection context
0925 07:26:07.315459 (+ 68us) client_negotiation.cc:240] Negotiation successful
0925 07:26:07.315557 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":84,"threads_started":1}
I20260925 07:26:07.316159 13641 heartbeater.cc:347] Connected to a master server at 127.5.158.126:40461
I20260925 07:26:07.316219 13641 heartbeater.cc:464] Registering TS with master...
I20260925 07:26:07.316327 13641 heartbeater.cc:511] Master 127.5.158.126:40461 requested a full tablet report, sending...
I20260925 07:26:07.316653 13298 ts_manager.cc:194] Registered new tserver with Master: 156cfffb10f04e838f2b80e19e59824a (127.5.158.66:36529)
I20260925 07:26:07.317035 13298 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.158.66@KRBTEST.COM'} at 127.5.158.66:41051
I20260925 07:26:07.321324 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 13510
I20260925 07:26:07.321460 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-1/wal/instance
I20260925 07:26:07.321672 5753 external_mini_cluster.cc:1428] Setting key 294c4aa88f7a0a7b8b90757631fc81df
I20260925 07:26:07.322544 13647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:07.317073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:40461 (local address 127.5.158.66:37315)
0925 07:26:07.317237 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:07.317245 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:07.317316 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:07.317443 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:07.317444 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:07.317639 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:07.317689 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:07.317690 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.318308 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.318309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:07.318823 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:07.318825 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.319178 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.319179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:07.319248 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:07.319702 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:07.319707 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:07.319969 (+ 262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:26:07.321435 (+ 1466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:26:07.321436 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:26:07.321436 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:26:07.321654 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:26:07.321873 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:26:07.321874 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:26:07.321874 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:26:07.321932 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:26:07.322255 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:26:07.322255 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:26:07.322296 (+ 41us) client_negotiation.cc:769] Sending connection context
0925 07:26:07.322371 (+ 75us) client_negotiation.cc:240] Negotiation successful
0925 07:26:07.322460 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":78,"threads_started":1}
I20260925 07:26:07.322554 13367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:07.317145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:37315 (local address 127.5.158.126:40461)
0925 07:26:07.317181 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:07.317183 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:07.317304 (+ 121us) server_negotiation.cc:407] Connection header received
0925 07:26:07.317374 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:07.317375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:07.317384 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:07.317438 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:07.317777 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.318217 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.318905 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.319106 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.319178 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:07.320076 (+ 898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:26:07.320077 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:26:07.320077 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:26:07.320083 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:26:07.321349 (+ 1266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:26:07.321746 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:26:07.321747 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:26:07.321748 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:26:07.321802 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:26:07.322003 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:26:07.322004 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:26:07.322004 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:26:07.322185 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:26:07.322252 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 07:26:07.322406 (+ 154us) server_negotiation.cc:299] Negotiation successful
0925 07:26:07.322468 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
WARNING: no policy specified for oryx/127.5.158.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.158.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.158.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/krb5kdc/oryx_127.5.158.67.keytab.
Entry for principal oryx/127.5.158.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/krb5kdc/oryx_127.5.158.67.keytab.
I20260925 07:26:07.345247 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.67:0
--local_ip_for_outbound_sockets=127.5.158.67
--webserver_interface=127.5.158.67
--webserver_port=0
--tserver_master_addrs=127.5.158.126:40461
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/krb5kdc/oryx_127.5.158.67.keytab
--principal=oryx/127.5.158.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:42619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:26:07.420341 13650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:26:07.420531 13650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:26:07.420567 13650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:26:07.421762 13650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:26:07.421824 13650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:26:07.421861 13650 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 07:26:07.421917 13650 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.67
I20260925 07:26:07.423267 13650 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:42619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/krb5kdc/oryx_127.5.158.67.keytab
--principal=oryx/127.5.158.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.158.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.158.126:40461
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13650
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.67
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:26:07.423488 13650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:26:07.423713 13650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:26:07.428398 13655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:26:07.428406 13656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:26:07.428524 13650 server_base.cc:1034] running on GCE node
W20260925 07:26:07.428411 13658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:26:07.428845 13650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:26:07.429016 13650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:26:07.430153 13650 hybrid_clock.cc:648] HybridClock initialized: now 1790321167430134 us; error 47 us; skew 500 ppm
Sep 25 07:26:07 dist-test-slave-1drq krb5kdc[13253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321167, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.158.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:26:07.432359 13650 init.cc:377] Logged in from keytab as oryx/127.5.158.67@KRBTEST.COM (short username oryx)
I20260925 07:26:07.432736 13650 webserver.cc:460] Webserver started at http://127.5.158.67:45023/ using document root <none> and password file <none>
I20260925 07:26:07.432934 13650 fs_manager.cc:359] Metadata directory not provided
I20260925 07:26:07.432974 13650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:26:07.433138 13650 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:26:07.434034 13650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-2/data/instance:
uuid: "9e74aea512cd41d1925f6b852e315c8f"
format_stamp: "Formatted at 2026-09-25 07:26:07 on dist-test-slave-1drq"
server_key: "9c425e8ce0926f55ebbfd85c9b5166c0"
server_key_iv: "8562df4a25a6fe69bb089afbb7fef34e"
server_key_version: "encryptionkey@0"
I20260925 07:26:07.434352 13650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-2/wal/instance:
uuid: "9e74aea512cd41d1925f6b852e315c8f"
format_stamp: "Formatted at 2026-09-25 07:26:07 on dist-test-slave-1drq"
server_key: "9c425e8ce0926f55ebbfd85c9b5166c0"
server_key_iv: "8562df4a25a6fe69bb089afbb7fef34e"
server_key_version: "encryptionkey@0"
I20260925 07:26:07.435590 13650 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 07:26:07.436226 13665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:07.436383 13650 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 07:26:07.436491 13650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-2/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-2/wal
uuid: "9e74aea512cd41d1925f6b852e315c8f"
format_stamp: "Formatted at 2026-09-25 07:26:07 on dist-test-slave-1drq"
server_key: "9c425e8ce0926f55ebbfd85c9b5166c0"
server_key_iv: "8562df4a25a6fe69bb089afbb7fef34e"
server_key_version: "encryptionkey@0"
I20260925 07:26:07.436571 13650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:26:07.446956 13650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:26:07.448535 13650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:26:07.448683 13650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:26:07.449170 13650 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:26:07.449229 13650 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:07.449262 13650 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:26:07.449319 13650 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 07:26:07 dist-test-slave-1drq krb5kdc[13253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321167, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.158.67@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM
I20260925 07:26:07.456100 13650 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.67:41489
I20260925 07:26:07.456174 13780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.67:41489 every 8 connection(s)
I20260925 07:26:07.456555 13650 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
I20260925 07:26:07.458303 13367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:07.449517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:43885 (local address 127.5.158.126:40461)
0925 07:26:07.449558 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:07.449560 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:07.449723 (+ 163us) server_negotiation.cc:407] Connection header received
0925 07:26:07.450166 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:07.450167 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:07.450180 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:07.450239 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:07.450787 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.451253 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.452172 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.452369 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.452450 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:07.455381 (+ 2931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:26:07.455382 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:26:07.455383 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:26:07.455392 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:26:07.457199 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:26:07.457595 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:26:07.457596 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:26:07.457596 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:26:07.457667 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:26:07.457883 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:26:07.457884 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:26:07.457884 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:26:07.457969 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:26:07.458012 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0925 07:26:07.458194 (+ 182us) server_negotiation.cc:299] Negotiation successful
0925 07:26:07.458247 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 07:26:07.458463 13674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:07.449509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:40461 (local address 127.5.158.67:43885)
0925 07:26:07.449622 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:07.449650 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:07.450067 (+ 417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:07.450316 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:07.450319 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:07.450515 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:07.450694 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:07.450699 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.451351 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.451352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:07.452088 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:07.452090 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.452456 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.452456 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:07.452577 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:07.453001 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:07.453011 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:07.455276 (+ 2265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:26:07.457282 (+ 2006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:26:07.457285 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:26:07.457287 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:26:07.457520 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:26:07.457719 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:26:07.457720 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:26:07.457720 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:26:07.457820 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:26:07.458004 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:26:07.458007 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:26:07.458098 (+ 91us) client_negotiation.cc:769] Sending connection context
0925 07:26:07.458171 (+ 73us) client_negotiation.cc:240] Negotiation successful
0925 07:26:07.458268 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":62,"threads_started":1}
I20260925 07:26:07.458928 13781 heartbeater.cc:347] Connected to a master server at 127.5.158.126:40461
I20260925 07:26:07.459022 13781 heartbeater.cc:464] Registering TS with master...
I20260925 07:26:07.459017 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 13650
I20260925 07:26:07.459147 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-2/wal/instance
I20260925 07:26:07.459182 13781 heartbeater.cc:511] Master 127.5.158.126:40461 requested a full tablet report, sending...
I20260925 07:26:07.459375 5753 external_mini_cluster.cc:1428] Setting key b66874a6cab8457fc195f276b17b4cea
I20260925 07:26:07.459630 13298 ts_manager.cc:194] Registered new tserver with Master: 9e74aea512cd41d1925f6b852e315c8f (127.5.158.67:41489)
I20260925 07:26:07.460089 13298 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.158.67@KRBTEST.COM'} at 127.5.158.67:43885
I20260925 07:26:07.460785 5753 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 07:26:07.466259 13787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:07.459923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:40461 (local address 127.5.158.67:34869)
0925 07:26:07.460059 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:07.460074 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:07.460135 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:07.460422 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:07.460422 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:07.460606 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:07.460670 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:07.460671 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.461431 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.461432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:07.462265 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:07.462267 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.462654 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.462655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:07.462721 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:07.463165 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:07.463170 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:07.463444 (+ 274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:26:07.465262 (+ 1818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:26:07.465263 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:26:07.465263 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:26:07.465404 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:26:07.465611 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:26:07.465612 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:26:07.465612 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:26:07.465667 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:26:07.465982 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:26:07.465983 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:26:07.466030 (+ 47us) client_negotiation.cc:769] Sending connection context
0925 07:26:07.466104 (+ 74us) client_negotiation.cc:240] Negotiation successful
0925 07:26:07.466195 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":61,"threads_started":1}
I20260925 07:26:07.466446 13367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:07.459988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:34869 (local address 127.5.158.126:40461)
0925 07:26:07.460027 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:07.460028 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:07.460178 (+ 150us) server_negotiation.cc:407] Connection header received
0925 07:26:07.460201 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:07.460202 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:07.460214 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:07.460291 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:07.460771 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.461337 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.462349 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.462550 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.462649 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:07.463553 (+ 904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:26:07.463555 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:26:07.463555 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:26:07.463562 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:26:07.465189 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:26:07.465477 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:26:07.465478 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:26:07.465478 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:26:07.465538 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:26:07.465741 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:26:07.465742 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:26:07.465742 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:26:07.465867 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:26:07.465949 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0925 07:26:07.466143 (+ 194us) server_negotiation.cc:299] Negotiation successful
0925 07:26:07.466223 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Sep 25 07:26:07 dist-test-slave-1drq krb5kdc[13253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321167, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 07:26:07 dist-test-slave-1drq krb5kdc[13253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 07:26:07 dist-test-slave-1drq krb5kdc[13253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
W20260925 07:26:07.483027 13367 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:26:07.478186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58670 (local address 127.5.158.126:40461)
0925 07:26:07.478235 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:07.478237 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:07.478246 (+ 9us) server_negotiation.cc:407] Connection header received
0925 07:26:07.478327 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:07.478328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:07.478337 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:07.478399 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:07.478971 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.479372 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.480105 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.480272 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.480345 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:07.482969 (+ 2624us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58670: BlockingRecv error: recv got EOF from 127.0.0.1:58670 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Sep 25 07:26:07 dist-test-slave-1drq krb5kdc[13253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321167, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM
I20260925 07:26:07.489401 13367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:07.483689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58676 (local address 127.5.158.126:40461)
0925 07:26:07.483727 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:07.483729 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:07.483790 (+ 61us) server_negotiation.cc:407] Connection header received
0925 07:26:07.483863 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:07.483864 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:07.483870 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:07.483917 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:07.484314 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.484719 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.485344 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.485499 (+ 155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.485546 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:07.487334 (+ 1788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:26:07.487336 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:26:07.487336 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:26:07.487341 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:26:07.488396 (+ 1055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:26:07.488668 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:26:07.488669 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:26:07.488669 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:26:07.488712 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:26:07.488871 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:26:07.488871 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:26:07.488872 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:26:07.488946 (+ 74us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:26:07.489014 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 07:26:07.489247 (+ 233us) server_negotiation.cc:299] Negotiation successful
0925 07:26:07.489321 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260925 07:26:07.490834 13298 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58676:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 07:26:07.492400 13298 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 07:26:07.498930 13804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:07.494643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:41851 (local address 127.0.0.1:44124)
0925 07:26:07.495129 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:07.495136 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:07.495199 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:07.495446 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:07.495446 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:07.495448 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:26:07.495494 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:07.495495 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.496664 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.496665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:07.497580 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:07.497583 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.498595 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.498596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:07.498703 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:07.498707 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 07:26:07.498782 (+ 75us) client_negotiation.cc:240] Negotiation successful
0925 07:26:07.498842 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"mutex_wait_us":2,"thread_start_us":58,"threads_started":1}
I20260925 07:26:07.498945 13805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:07.494748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44124 (local address 127.5.158.65:41851)
0925 07:26:07.494891 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:07.494897 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:07.495199 (+ 302us) server_negotiation.cc:407] Connection header received
0925 07:26:07.495274 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:07.495276 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:07.495291 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:07.495359 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:26:07.495561 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.496584 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.497698 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.498503 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.498583 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:07.498658 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 07:26:07.498804 (+ 146us) server_negotiation.cc:299] Negotiation successful
0925 07:26:07.498843 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":85,"threads_started":1}
I20260925 07:26:07.499168 13802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:07.494537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32862 (local address 127.5.158.67:41489)
0925 07:26:07.495302 (+ 765us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:07.495306 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:07.495316 (+ 10us) server_negotiation.cc:407] Connection header received
0925 07:26:07.495342 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:07.495344 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:07.495356 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:07.495433 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:26:07.495593 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.496243 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.497800 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.498503 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.498583 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:07.498664 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0925 07:26:07.499039 (+ 375us) server_negotiation.cc:299] Negotiation successful
0925 07:26:07.499073 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":739,"thread_start_us":68,"threads_started":1}
I20260925 07:26:07.498929 13800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:07.494468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.67:41489 (local address 127.0.0.1:32862)
0925 07:26:07.494724 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:07.494756 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:07.494811 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:07.495492 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:07.495493 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:07.495495 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:26:07.495526 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:07.495527 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.496326 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.496327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:07.497703 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:07.497705 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.498593 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.498595 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:07.498693 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:07.498707 (+ 14us) client_negotiation.cc:769] Sending connection context
0925 07:26:07.498782 (+ 75us) client_negotiation.cc:240] Negotiation successful
0925 07:26:07.498843 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"mutex_wait_us":104,"thread_start_us":49,"threads_started":1}
I20260925 07:26:07.499799 13801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:07.494470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.66:36529 (local address 127.0.0.1:48152)
0925 07:26:07.494873 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:07.494879 (+ 6us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:07.494901 (+ 22us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:07.495095 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:07.495097 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:07.495098 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:26:07.495145 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:07.495146 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.496399 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.496400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:07.497703 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:07.497705 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.499565 (+ 1860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.499566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:07.499630 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:07.499634 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 07:26:07.499695 (+ 61us) client_negotiation.cc:240] Negotiation successful
0925 07:26:07.499737 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":397,"mutex_wait_us":70,"thread_start_us":52,"threads_started":1}
I20260925 07:26:07.499697 13435 tablet_service.cc:1467] Processing CreateTablet for tablet ed9a3ffdcbef4406992a99339c6ee399 (DEFAULT_TABLE table=test-table [id=f043c38ab599447fb0db45212884c736]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:26:07.499852 13803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:07.494569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48152 (local address 127.5.158.66:36529)
0925 07:26:07.494941 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:07.494945 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:07.494954 (+ 9us) server_negotiation.cc:407] Connection header received
0925 07:26:07.494994 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:07.494995 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:07.495006 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:07.495070 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:26:07.495228 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.496335 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.498697 (+ 2362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.499467 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.499556 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:07.499625 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 07:26:07.499717 (+ 92us) server_negotiation.cc:299] Negotiation successful
0925 07:26:07.499759 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":67,"threads_started":1}
I20260925 07:26:07.500036 13435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed9a3ffdcbef4406992a99339c6ee399. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:26:07.500438 13575 tablet_service.cc:1467] Processing CreateTablet for tablet ed9a3ffdcbef4406992a99339c6ee399 (DEFAULT_TABLE table=test-table [id=f043c38ab599447fb0db45212884c736]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:26:07.500756 13575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed9a3ffdcbef4406992a99339c6ee399. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:26:07.501657 13715 tablet_service.cc:1467] Processing CreateTablet for tablet ed9a3ffdcbef4406992a99339c6ee399 (DEFAULT_TABLE table=test-table [id=f043c38ab599447fb0db45212884c736]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:26:07.501950 13715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed9a3ffdcbef4406992a99339c6ee399. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:26:07.504009 13806 tablet_bootstrap.cc:492] T ed9a3ffdcbef4406992a99339c6ee399 P 1353649238cf4f7a909ccd8e1462cecd: Bootstrap starting.
I20260925 07:26:07.504509 13807 tablet_bootstrap.cc:492] T ed9a3ffdcbef4406992a99339c6ee399 P 156cfffb10f04e838f2b80e19e59824a: Bootstrap starting.
I20260925 07:26:07.504856 13806 tablet_bootstrap.cc:654] T ed9a3ffdcbef4406992a99339c6ee399 P 1353649238cf4f7a909ccd8e1462cecd: Neither blocks nor log segments found. Creating new log.
I20260925 07:26:07.505203 13806 log.cc:824] T ed9a3ffdcbef4406992a99339c6ee399 P 1353649238cf4f7a909ccd8e1462cecd: Log is configured to *not* fsync() on all Append() calls
I20260925 07:26:07.505241 13807 tablet_bootstrap.cc:654] T ed9a3ffdcbef4406992a99339c6ee399 P 156cfffb10f04e838f2b80e19e59824a: Neither blocks nor log segments found. Creating new log.
I20260925 07:26:07.505505 13808 tablet_bootstrap.cc:492] T ed9a3ffdcbef4406992a99339c6ee399 P 9e74aea512cd41d1925f6b852e315c8f: Bootstrap starting.
I20260925 07:26:07.505560 13807 log.cc:824] T ed9a3ffdcbef4406992a99339c6ee399 P 156cfffb10f04e838f2b80e19e59824a: Log is configured to *not* fsync() on all Append() calls
I20260925 07:26:07.505883 13806 tablet_bootstrap.cc:492] T ed9a3ffdcbef4406992a99339c6ee399 P 1353649238cf4f7a909ccd8e1462cecd: No bootstrap required, opened a new log
I20260925 07:26:07.505951 13806 ts_tablet_manager.cc:1397] T ed9a3ffdcbef4406992a99339c6ee399 P 1353649238cf4f7a909ccd8e1462cecd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 07:26:07.506242 13807 tablet_bootstrap.cc:492] T ed9a3ffdcbef4406992a99339c6ee399 P 156cfffb10f04e838f2b80e19e59824a: No bootstrap required, opened a new log
I20260925 07:26:07.506286 13808 tablet_bootstrap.cc:654] T ed9a3ffdcbef4406992a99339c6ee399 P 9e74aea512cd41d1925f6b852e315c8f: Neither blocks nor log segments found. Creating new log.
I20260925 07:26:07.506326 13807 ts_tablet_manager.cc:1397] T ed9a3ffdcbef4406992a99339c6ee399 P 156cfffb10f04e838f2b80e19e59824a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260925 07:26:07.506601 13808 log.cc:824] T ed9a3ffdcbef4406992a99339c6ee399 P 9e74aea512cd41d1925f6b852e315c8f: Log is configured to *not* fsync() on all Append() calls
I20260925 07:26:07.507380 13808 tablet_bootstrap.cc:492] T ed9a3ffdcbef4406992a99339c6ee399 P 9e74aea512cd41d1925f6b852e315c8f: No bootstrap required, opened a new log
I20260925 07:26:07.507447 13808 ts_tablet_manager.cc:1397] T ed9a3ffdcbef4406992a99339c6ee399 P 9e74aea512cd41d1925f6b852e315c8f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 07:26:07.508082 13807 raft_consensus.cc:348] T ed9a3ffdcbef4406992a99339c6ee399 P 156cfffb10f04e838f2b80e19e59824a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } } peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } }
I20260925 07:26:07.508036 13806 raft_consensus.cc:348] T ed9a3ffdcbef4406992a99339c6ee399 P 1353649238cf4f7a909ccd8e1462cecd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } } peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } }
I20260925 07:26:07.508227 13806 raft_consensus.cc:374] T ed9a3ffdcbef4406992a99339c6ee399 P 1353649238cf4f7a909ccd8e1462cecd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:26:07.508229 13807 raft_consensus.cc:374] T ed9a3ffdcbef4406992a99339c6ee399 P 156cfffb10f04e838f2b80e19e59824a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:26:07.508272 13807 raft_consensus.cc:729] T ed9a3ffdcbef4406992a99339c6ee399 P 156cfffb10f04e838f2b80e19e59824a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 156cfffb10f04e838f2b80e19e59824a, State: Initialized, Role: FOLLOWER
I20260925 07:26:07.508272 13806 raft_consensus.cc:729] T ed9a3ffdcbef4406992a99339c6ee399 P 1353649238cf4f7a909ccd8e1462cecd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1353649238cf4f7a909ccd8e1462cecd, State: Initialized, Role: FOLLOWER
I20260925 07:26:07.508379 13807 consensus_queue.cc:260] T ed9a3ffdcbef4406992a99339c6ee399 P 156cfffb10f04e838f2b80e19e59824a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } } peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } }
I20260925 07:26:07.508399 13806 consensus_queue.cc:260] T ed9a3ffdcbef4406992a99339c6ee399 P 1353649238cf4f7a909ccd8e1462cecd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } } peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } }
I20260925 07:26:07.508642 13806 ts_tablet_manager.cc:1428] T ed9a3ffdcbef4406992a99339c6ee399 P 1353649238cf4f7a909ccd8e1462cecd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 07:26:07.508679 13807 ts_tablet_manager.cc:1428] T ed9a3ffdcbef4406992a99339c6ee399 P 156cfffb10f04e838f2b80e19e59824a: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260925 07:26:07.508746 13501 heartbeater.cc:503] Master 127.5.158.126:40461 was elected leader, sending a full tablet report...
I20260925 07:26:07.508695 13641 heartbeater.cc:503] Master 127.5.158.126:40461 was elected leader, sending a full tablet report...
I20260925 07:26:07.509394 13808 raft_consensus.cc:348] T ed9a3ffdcbef4406992a99339c6ee399 P 9e74aea512cd41d1925f6b852e315c8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } } peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } }
I20260925 07:26:07.509532 13808 raft_consensus.cc:374] T ed9a3ffdcbef4406992a99339c6ee399 P 9e74aea512cd41d1925f6b852e315c8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:26:07.509569 13808 raft_consensus.cc:729] T ed9a3ffdcbef4406992a99339c6ee399 P 9e74aea512cd41d1925f6b852e315c8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e74aea512cd41d1925f6b852e315c8f, State: Initialized, Role: FOLLOWER
I20260925 07:26:07.509688 13808 consensus_queue.cc:260] T ed9a3ffdcbef4406992a99339c6ee399 P 9e74aea512cd41d1925f6b852e315c8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } } peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } }
I20260925 07:26:07.509981 13808 ts_tablet_manager.cc:1428] T ed9a3ffdcbef4406992a99339c6ee399 P 9e74aea512cd41d1925f6b852e315c8f: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260925 07:26:07.510102 13781 heartbeater.cc:503] Master 127.5.158.126:40461 was elected leader, sending a full tablet report...
I20260925 07:26:07.534323 13812 raft_consensus.cc:482] T ed9a3ffdcbef4406992a99339c6ee399 P 1353649238cf4f7a909ccd8e1462cecd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 07:26:07.534397 13812 raft_consensus.cc:504] T ed9a3ffdcbef4406992a99339c6ee399 P 1353649238cf4f7a909ccd8e1462cecd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } } peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } }
I20260925 07:26:07.534716 13812 leader_election.cc:288] T ed9a3ffdcbef4406992a99339c6ee399 P 1353649238cf4f7a909ccd8e1462cecd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 156cfffb10f04e838f2b80e19e59824a (127.5.158.66:36529), 9e74aea512cd41d1925f6b852e315c8f (127.5.158.67:41489)
I20260925 07:26:07.538381 13802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:07.535016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:41885 (local address 127.5.158.67:41489)
0925 07:26:07.535060 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:07.535062 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:07.535168 (+ 106us) server_negotiation.cc:407] Connection header received
0925 07:26:07.535219 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:07.535220 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:07.535227 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:07.535294 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:26:07.535431 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.536107 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.537439 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.538016 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.538078 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:07.538086 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 07:26:07.538281 (+ 195us) server_negotiation.cc:299] Negotiation successful
0925 07:26:07.538318 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260925 07:26:07.538380 13803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:07.534981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:40121 (local address 127.5.158.66:36529)
0925 07:26:07.535026 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:07.535028 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:07.535038 (+ 10us) server_negotiation.cc:407] Connection header received
0925 07:26:07.535067 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:07.535068 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:07.535077 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:07.535134 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:26:07.535282 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.536043 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.537439 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.537990 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.538082 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:07.538088 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0925 07:26:07.538281 (+ 193us) server_negotiation.cc:299] Negotiation successful
0925 07:26:07.538318 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 07:26:07.538429 13815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:07.534881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.67:41489 (local address 127.5.158.65:41885)
0925 07:26:07.535100 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:07.535109 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:07.535171 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:07.535308 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:07.535308 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:07.535309 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:26:07.535372 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:07.535373 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.536193 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.536193 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:07.537363 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:07.537365 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.538089 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.538089 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:07.538200 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:07.538207 (+ 7us) client_negotiation.cc:769] Sending connection context
0925 07:26:07.538264 (+ 57us) client_negotiation.cc:240] Negotiation successful
0925 07:26:07.538315 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"mutex_wait_us":104,"thread_start_us":80,"threads_started":1}
I20260925 07:26:07.538435 13394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:07.534881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.66:36529 (local address 127.5.158.65:40121)
0925 07:26:07.534956 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:07.534968 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:07.535018 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:07.535141 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:07.535142 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:07.535143 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:26:07.535212 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:07.535214 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.536115 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.536115 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:07.537353 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:07.537356 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.538085 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.538086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:07.538200 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:07.538206 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 07:26:07.538264 (+ 58us) client_negotiation.cc:240] Negotiation successful
0925 07:26:07.538315 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":34}
I20260925 07:26:07.538990 13595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9a3ffdcbef4406992a99339c6ee399" candidate_uuid: "1353649238cf4f7a909ccd8e1462cecd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "156cfffb10f04e838f2b80e19e59824a" is_pre_election: true
I20260925 07:26:07.539009 13735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9a3ffdcbef4406992a99339c6ee399" candidate_uuid: "1353649238cf4f7a909ccd8e1462cecd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e74aea512cd41d1925f6b852e315c8f" is_pre_election: true
I20260925 07:26:07.539144 13735 raft_consensus.cc:2454] T ed9a3ffdcbef4406992a99339c6ee399 P 9e74aea512cd41d1925f6b852e315c8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1353649238cf4f7a909ccd8e1462cecd in term 0.
I20260925 07:26:07.539141 13595 raft_consensus.cc:2454] T ed9a3ffdcbef4406992a99339c6ee399 P 156cfffb10f04e838f2b80e19e59824a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1353649238cf4f7a909ccd8e1462cecd in term 0.
I20260925 07:26:07.539404 13389 leader_election.cc:302] T ed9a3ffdcbef4406992a99339c6ee399 P 1353649238cf4f7a909ccd8e1462cecd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1353649238cf4f7a909ccd8e1462cecd, 9e74aea512cd41d1925f6b852e315c8f; no voters:
I20260925 07:26:07.539549 13812 raft_consensus.cc:2781] T ed9a3ffdcbef4406992a99339c6ee399 P 1353649238cf4f7a909ccd8e1462cecd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 07:26:07.539618 13812 raft_consensus.cc:482] T ed9a3ffdcbef4406992a99339c6ee399 P 1353649238cf4f7a909ccd8e1462cecd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 07:26:07.539651 13812 raft_consensus.cc:3037] T ed9a3ffdcbef4406992a99339c6ee399 P 1353649238cf4f7a909ccd8e1462cecd [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:26:07.540624 13812 raft_consensus.cc:504] T ed9a3ffdcbef4406992a99339c6ee399 P 1353649238cf4f7a909ccd8e1462cecd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } } peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } }
I20260925 07:26:07.540760 13812 leader_election.cc:288] T ed9a3ffdcbef4406992a99339c6ee399 P 1353649238cf4f7a909ccd8e1462cecd [CANDIDATE]: Term 1 election: Requested vote from peers 156cfffb10f04e838f2b80e19e59824a (127.5.158.66:36529), 9e74aea512cd41d1925f6b852e315c8f (127.5.158.67:41489)
I20260925 07:26:07.540886 13735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9a3ffdcbef4406992a99339c6ee399" candidate_uuid: "1353649238cf4f7a909ccd8e1462cecd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e74aea512cd41d1925f6b852e315c8f"
I20260925 07:26:07.540884 13595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9a3ffdcbef4406992a99339c6ee399" candidate_uuid: "1353649238cf4f7a909ccd8e1462cecd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "156cfffb10f04e838f2b80e19e59824a"
I20260925 07:26:07.540956 13595 raft_consensus.cc:3037] T ed9a3ffdcbef4406992a99339c6ee399 P 156cfffb10f04e838f2b80e19e59824a [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:26:07.540975 13735 raft_consensus.cc:3037] T ed9a3ffdcbef4406992a99339c6ee399 P 9e74aea512cd41d1925f6b852e315c8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:26:07.541613 13595 raft_consensus.cc:2454] T ed9a3ffdcbef4406992a99339c6ee399 P 156cfffb10f04e838f2b80e19e59824a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1353649238cf4f7a909ccd8e1462cecd in term 1.
I20260925 07:26:07.541621 13735 raft_consensus.cc:2454] T ed9a3ffdcbef4406992a99339c6ee399 P 9e74aea512cd41d1925f6b852e315c8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1353649238cf4f7a909ccd8e1462cecd in term 1.
I20260925 07:26:07.541767 13389 leader_election.cc:302] T ed9a3ffdcbef4406992a99339c6ee399 P 1353649238cf4f7a909ccd8e1462cecd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1353649238cf4f7a909ccd8e1462cecd, 9e74aea512cd41d1925f6b852e315c8f; no voters:
I20260925 07:26:07.541875 13812 raft_consensus.cc:2781] T ed9a3ffdcbef4406992a99339c6ee399 P 1353649238cf4f7a909ccd8e1462cecd [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:26:07.541990 13812 raft_consensus.cc:686] T ed9a3ffdcbef4406992a99339c6ee399 P 1353649238cf4f7a909ccd8e1462cecd [term 1 LEADER]: Becoming Leader. State: Replica: 1353649238cf4f7a909ccd8e1462cecd, State: Running, Role: LEADER
I20260925 07:26:07.542074 13812 consensus_queue.cc:237] T ed9a3ffdcbef4406992a99339c6ee399 P 1353649238cf4f7a909ccd8e1462cecd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } } peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } }
I20260925 07:26:07.542829 13297 catalog_manager.cc:5495] T ed9a3ffdcbef4406992a99339c6ee399 P 1353649238cf4f7a909ccd8e1462cecd reported cstate change: term changed from 0 to 1, leader changed from <none> to 1353649238cf4f7a909ccd8e1462cecd (127.5.158.65). New cstate: current_term: 1 leader_uuid: "1353649238cf4f7a909ccd8e1462cecd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } health_report { overall_health: HEALTHY } } }
I20260925 07:26:07.554791 13805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:07.552701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44128 (local address 127.5.158.65:41851)
0925 07:26:07.552752 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:07.552754 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:07.552762 (+ 8us) server_negotiation.cc:407] Connection header received
0925 07:26:07.552791 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:07.552791 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:07.552804 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:07.552879 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 07:26:07.552974 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.553419 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.554112 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.554303 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.554370 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:07.554530 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 07:26:07.554624 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 07:26:07.554698 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 07:26:07.554717 (+ 19us) server_negotiation.cc:299] Negotiation successful
0925 07:26:07.554733 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 07:26:07.556128 13595 raft_consensus.cc:1261] T ed9a3ffdcbef4406992a99339c6ee399 P 156cfffb10f04e838f2b80e19e59824a [term 1 FOLLOWER]: Refusing update from remote peer 1353649238cf4f7a909ccd8e1462cecd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:26:07.556193 13735 raft_consensus.cc:1261] T ed9a3ffdcbef4406992a99339c6ee399 P 9e74aea512cd41d1925f6b852e315c8f [term 1 FOLLOWER]: Refusing update from remote peer 1353649238cf4f7a909ccd8e1462cecd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:26:07.556408 13816 consensus_queue.cc:1035] T ed9a3ffdcbef4406992a99339c6ee399 P 1353649238cf4f7a909ccd8e1462cecd [LEADER]: Connected to new peer: Peer: permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:26:07.556592 13812 consensus_queue.cc:1035] T ed9a3ffdcbef4406992a99339c6ee399 P 1353649238cf4f7a909ccd8e1462cecd [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:26:07.558759 13819 mvcc.cc:205] Tried to move back new op lower bound from 7333155502308048896 to 7333155502252941312. Current Snapshot: MvccSnapshot[applied={T|T < 7333155502308048896}]
I20260925 07:26:07.560995 13298 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58676:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 07:26:07.561082 13298 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58676:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 07:26:07.561714 13298 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f5e596c5f3284f62a2c097cde2041a96: Sending DeleteTablet for 3 replicas of tablet ed9a3ffdcbef4406992a99339c6ee399
I20260925 07:26:07.561986 13575 tablet_service.cc:1514] Processing DeleteTablet for tablet ed9a3ffdcbef4406992a99339c6ee399 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:26:07 UTC) from {username='oryx', principal='oryx/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:48152
I20260925 07:26:07.562002 13715 tablet_service.cc:1514] Processing DeleteTablet for tablet ed9a3ffdcbef4406992a99339c6ee399 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:26:07 UTC) from {username='oryx', principal='oryx/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:32862
I20260925 07:26:07.562305 13435 tablet_service.cc:1514] Processing DeleteTablet for tablet ed9a3ffdcbef4406992a99339c6ee399 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:26:07 UTC) from {username='oryx', principal='oryx/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:44124
I20260925 07:26:07.562472 13298 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58676:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 07:26:07.562585 13298 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 07:26:07.562783 13829 tablet_replica.cc:330] T ed9a3ffdcbef4406992a99339c6ee399 P 9e74aea512cd41d1925f6b852e315c8f: stopping tablet replica
I20260925 07:26:07.562872 13829 raft_consensus.cc:2229] T ed9a3ffdcbef4406992a99339c6ee399 P 9e74aea512cd41d1925f6b852e315c8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 07:26:07.562976 13829 raft_consensus.cc:2258] T ed9a3ffdcbef4406992a99339c6ee399 P 9e74aea512cd41d1925f6b852e315c8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:26:07.563113 13828 tablet_replica.cc:330] T ed9a3ffdcbef4406992a99339c6ee399 P 156cfffb10f04e838f2b80e19e59824a: stopping tablet replica
I20260925 07:26:07.563211 13828 raft_consensus.cc:2229] T ed9a3ffdcbef4406992a99339c6ee399 P 156cfffb10f04e838f2b80e19e59824a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 07:26:07.563323 13828 raft_consensus.cc:2258] T ed9a3ffdcbef4406992a99339c6ee399 P 156cfffb10f04e838f2b80e19e59824a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:26:07.563489 13830 tablet_replica.cc:330] T ed9a3ffdcbef4406992a99339c6ee399 P 1353649238cf4f7a909ccd8e1462cecd: stopping tablet replica
I20260925 07:26:07.563577 13830 raft_consensus.cc:2229] T ed9a3ffdcbef4406992a99339c6ee399 P 1353649238cf4f7a909ccd8e1462cecd [term 1 LEADER]: Raft consensus shutting down.
I20260925 07:26:07.563601 13829 ts_tablet_manager.cc:1905] T ed9a3ffdcbef4406992a99339c6ee399 P 9e74aea512cd41d1925f6b852e315c8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:26:07.563680 13830 raft_consensus.cc:2258] T ed9a3ffdcbef4406992a99339c6ee399 P 1353649238cf4f7a909ccd8e1462cecd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:26:07.564154 13830 ts_tablet_manager.cc:1905] T ed9a3ffdcbef4406992a99339c6ee399 P 1353649238cf4f7a909ccd8e1462cecd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:26:07.564172 13715 tablet_service.cc:1467] Processing CreateTablet for tablet e29d07c10c374ade95f8c9d1ea388e30 (DEFAULT_TABLE table=test-table [id=bf752289d9d945609cba519cf234d148]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:26:07.564260 13715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e29d07c10c374ade95f8c9d1ea388e30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:26:07.565536 13808 tablet_bootstrap.cc:492] T e29d07c10c374ade95f8c9d1ea388e30 P 9e74aea512cd41d1925f6b852e315c8f: Bootstrap starting.
I20260925 07:26:07.565768 13830 ts_tablet_manager.cc:1918] T ed9a3ffdcbef4406992a99339c6ee399 P 1353649238cf4f7a909ccd8e1462cecd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 07:26:07.565834 13830 log.cc:1196] T ed9a3ffdcbef4406992a99339c6ee399 P 1353649238cf4f7a909ccd8e1462cecd: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-0/wal/wals/ed9a3ffdcbef4406992a99339c6ee399
I20260925 07:26:07.565913 13808 tablet_bootstrap.cc:654] T e29d07c10c374ade95f8c9d1ea388e30 P 9e74aea512cd41d1925f6b852e315c8f: Neither blocks nor log segments found. Creating new log.
I20260925 07:26:07.565927 13829 ts_tablet_manager.cc:1918] T ed9a3ffdcbef4406992a99339c6ee399 P 9e74aea512cd41d1925f6b852e315c8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 07:26:07.566083 13830 ts_tablet_manager.cc:1939] T ed9a3ffdcbef4406992a99339c6ee399 P 1353649238cf4f7a909ccd8e1462cecd: Deleting consensus metadata
I20260925 07:26:07.566102 13829 log.cc:1196] T ed9a3ffdcbef4406992a99339c6ee399 P 9e74aea512cd41d1925f6b852e315c8f: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-2/wal/wals/ed9a3ffdcbef4406992a99339c6ee399
I20260925 07:26:07.566406 13285 catalog_manager.cc:4841] TS 1353649238cf4f7a909ccd8e1462cecd (127.5.158.65:41851): tablet ed9a3ffdcbef4406992a99339c6ee399 (table test-table [id=f043c38ab599447fb0db45212884c736]) successfully deleted
I20260925 07:26:07.566557 13829 ts_tablet_manager.cc:1939] T ed9a3ffdcbef4406992a99339c6ee399 P 9e74aea512cd41d1925f6b852e315c8f: Deleting consensus metadata
I20260925 07:26:07.566569 13575 tablet_service.cc:1467] Processing CreateTablet for tablet e29d07c10c374ade95f8c9d1ea388e30 (DEFAULT_TABLE table=test-table [id=bf752289d9d945609cba519cf234d148]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:26:07.566676 13575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e29d07c10c374ade95f8c9d1ea388e30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:26:07.566955 13285 catalog_manager.cc:4841] TS 9e74aea512cd41d1925f6b852e315c8f (127.5.158.67:41489): tablet ed9a3ffdcbef4406992a99339c6ee399 (table test-table [id=f043c38ab599447fb0db45212884c736]) successfully deleted
I20260925 07:26:07.567906 13807 tablet_bootstrap.cc:492] T e29d07c10c374ade95f8c9d1ea388e30 P 156cfffb10f04e838f2b80e19e59824a: Bootstrap starting.
I20260925 07:26:07.568068 13435 tablet_service.cc:1467] Processing CreateTablet for tablet e29d07c10c374ade95f8c9d1ea388e30 (DEFAULT_TABLE table=test-table [id=bf752289d9d945609cba519cf234d148]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:26:07.568176 13435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e29d07c10c374ade95f8c9d1ea388e30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:26:07.568288 13807 tablet_bootstrap.cc:654] T e29d07c10c374ade95f8c9d1ea388e30 P 156cfffb10f04e838f2b80e19e59824a: Neither blocks nor log segments found. Creating new log.
I20260925 07:26:07.568854 13807 tablet_bootstrap.cc:492] T e29d07c10c374ade95f8c9d1ea388e30 P 156cfffb10f04e838f2b80e19e59824a: No bootstrap required, opened a new log
I20260925 07:26:07.568907 13807 ts_tablet_manager.cc:1397] T e29d07c10c374ade95f8c9d1ea388e30 P 156cfffb10f04e838f2b80e19e59824a: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260925 07:26:07.569031 13807 raft_consensus.cc:348] T e29d07c10c374ade95f8c9d1ea388e30 P 156cfffb10f04e838f2b80e19e59824a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } } peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } }
I20260925 07:26:07.569131 13807 raft_consensus.cc:374] T e29d07c10c374ade95f8c9d1ea388e30 P 156cfffb10f04e838f2b80e19e59824a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:26:07.569123 13806 tablet_bootstrap.cc:492] T e29d07c10c374ade95f8c9d1ea388e30 P 1353649238cf4f7a909ccd8e1462cecd: Bootstrap starting.
I20260925 07:26:07.569154 13807 raft_consensus.cc:729] T e29d07c10c374ade95f8c9d1ea388e30 P 156cfffb10f04e838f2b80e19e59824a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 156cfffb10f04e838f2b80e19e59824a, State: Initialized, Role: FOLLOWER
I20260925 07:26:07.569192 13807 consensus_queue.cc:260] T e29d07c10c374ade95f8c9d1ea388e30 P 156cfffb10f04e838f2b80e19e59824a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } } peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } }
I20260925 07:26:07.569275 13807 ts_tablet_manager.cc:1428] T e29d07c10c374ade95f8c9d1ea388e30 P 156cfffb10f04e838f2b80e19e59824a: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260925 07:26:07.569473 13806 tablet_bootstrap.cc:654] T e29d07c10c374ade95f8c9d1ea388e30 P 1353649238cf4f7a909ccd8e1462cecd: Neither blocks nor log segments found. Creating new log.
I20260925 07:26:07.569545 13808 tablet_bootstrap.cc:492] T e29d07c10c374ade95f8c9d1ea388e30 P 9e74aea512cd41d1925f6b852e315c8f: No bootstrap required, opened a new log
I20260925 07:26:07.569588 13808 ts_tablet_manager.cc:1397] T e29d07c10c374ade95f8c9d1ea388e30 P 9e74aea512cd41d1925f6b852e315c8f: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260925 07:26:07.569715 13808 raft_consensus.cc:348] T e29d07c10c374ade95f8c9d1ea388e30 P 9e74aea512cd41d1925f6b852e315c8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } } peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } }
I20260925 07:26:07.569773 13808 raft_consensus.cc:374] T e29d07c10c374ade95f8c9d1ea388e30 P 9e74aea512cd41d1925f6b852e315c8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:26:07.569794 13808 raft_consensus.cc:729] T e29d07c10c374ade95f8c9d1ea388e30 P 9e74aea512cd41d1925f6b852e315c8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e74aea512cd41d1925f6b852e315c8f, State: Initialized, Role: FOLLOWER
I20260925 07:26:07.569833 13808 consensus_queue.cc:260] T e29d07c10c374ade95f8c9d1ea388e30 P 9e74aea512cd41d1925f6b852e315c8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } } peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } }
I20260925 07:26:07.569904 13808 ts_tablet_manager.cc:1428] T e29d07c10c374ade95f8c9d1ea388e30 P 9e74aea512cd41d1925f6b852e315c8f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:07.570011 13806 tablet_bootstrap.cc:492] T e29d07c10c374ade95f8c9d1ea388e30 P 1353649238cf4f7a909ccd8e1462cecd: No bootstrap required, opened a new log
I20260925 07:26:07.570058 13806 ts_tablet_manager.cc:1397] T e29d07c10c374ade95f8c9d1ea388e30 P 1353649238cf4f7a909ccd8e1462cecd: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 07:26:07.570204 13806 raft_consensus.cc:348] T e29d07c10c374ade95f8c9d1ea388e30 P 1353649238cf4f7a909ccd8e1462cecd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } } peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } }
I20260925 07:26:07.570286 13806 raft_consensus.cc:374] T e29d07c10c374ade95f8c9d1ea388e30 P 1353649238cf4f7a909ccd8e1462cecd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:26:07.570313 13806 raft_consensus.cc:729] T e29d07c10c374ade95f8c9d1ea388e30 P 1353649238cf4f7a909ccd8e1462cecd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1353649238cf4f7a909ccd8e1462cecd, State: Initialized, Role: FOLLOWER
I20260925 07:26:07.570355 13806 consensus_queue.cc:260] T e29d07c10c374ade95f8c9d1ea388e30 P 1353649238cf4f7a909ccd8e1462cecd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } } peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } }
I20260925 07:26:07.570446 13806 ts_tablet_manager.cc:1428] T e29d07c10c374ade95f8c9d1ea388e30 P 1353649238cf4f7a909ccd8e1462cecd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:07.573295 13828 ts_tablet_manager.cc:1905] T ed9a3ffdcbef4406992a99339c6ee399 P 156cfffb10f04e838f2b80e19e59824a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:26:07.574374 13828 ts_tablet_manager.cc:1918] T ed9a3ffdcbef4406992a99339c6ee399 P 156cfffb10f04e838f2b80e19e59824a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 07:26:07.574445 13828 log.cc:1196] T ed9a3ffdcbef4406992a99339c6ee399 P 156cfffb10f04e838f2b80e19e59824a: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-1/wal/wals/ed9a3ffdcbef4406992a99339c6ee399
I20260925 07:26:07.574640 13828 ts_tablet_manager.cc:1939] T ed9a3ffdcbef4406992a99339c6ee399 P 156cfffb10f04e838f2b80e19e59824a: Deleting consensus metadata
I20260925 07:26:07.574993 13284 catalog_manager.cc:4841] TS 156cfffb10f04e838f2b80e19e59824a (127.5.158.66:36529): tablet ed9a3ffdcbef4406992a99339c6ee399 (table test-table [id=f043c38ab599447fb0db45212884c736]) successfully deleted
W20260925 07:26:07.661764 13502 tablet.cc:2369] T e29d07c10c374ade95f8c9d1ea388e30 P 1353649238cf4f7a909ccd8e1462cecd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 07:26:07.682557 13816 raft_consensus.cc:482] T e29d07c10c374ade95f8c9d1ea388e30 P 1353649238cf4f7a909ccd8e1462cecd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 07:26:07.682646 13816 raft_consensus.cc:504] T e29d07c10c374ade95f8c9d1ea388e30 P 1353649238cf4f7a909ccd8e1462cecd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } } peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } }
I20260925 07:26:07.682819 13816 leader_election.cc:288] T e29d07c10c374ade95f8c9d1ea388e30 P 1353649238cf4f7a909ccd8e1462cecd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 156cfffb10f04e838f2b80e19e59824a (127.5.158.66:36529), 9e74aea512cd41d1925f6b852e315c8f (127.5.158.67:41489)
I20260925 07:26:07.683058 13595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e29d07c10c374ade95f8c9d1ea388e30" candidate_uuid: "1353649238cf4f7a909ccd8e1462cecd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "156cfffb10f04e838f2b80e19e59824a" is_pre_election: true
I20260925 07:26:07.683033 13735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e29d07c10c374ade95f8c9d1ea388e30" candidate_uuid: "1353649238cf4f7a909ccd8e1462cecd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e74aea512cd41d1925f6b852e315c8f" is_pre_election: true
I20260925 07:26:07.683159 13595 raft_consensus.cc:2454] T e29d07c10c374ade95f8c9d1ea388e30 P 156cfffb10f04e838f2b80e19e59824a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1353649238cf4f7a909ccd8e1462cecd in term 0.
I20260925 07:26:07.683159 13735 raft_consensus.cc:2454] T e29d07c10c374ade95f8c9d1ea388e30 P 9e74aea512cd41d1925f6b852e315c8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1353649238cf4f7a909ccd8e1462cecd in term 0.
I20260925 07:26:07.683324 13389 leader_election.cc:302] T e29d07c10c374ade95f8c9d1ea388e30 P 1353649238cf4f7a909ccd8e1462cecd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1353649238cf4f7a909ccd8e1462cecd, 9e74aea512cd41d1925f6b852e315c8f; no voters:
I20260925 07:26:07.683430 13816 raft_consensus.cc:2781] T e29d07c10c374ade95f8c9d1ea388e30 P 1353649238cf4f7a909ccd8e1462cecd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 07:26:07.683478 13816 raft_consensus.cc:482] T e29d07c10c374ade95f8c9d1ea388e30 P 1353649238cf4f7a909ccd8e1462cecd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 07:26:07.683507 13816 raft_consensus.cc:3037] T e29d07c10c374ade95f8c9d1ea388e30 P 1353649238cf4f7a909ccd8e1462cecd [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:26:07.684226 13816 raft_consensus.cc:504] T e29d07c10c374ade95f8c9d1ea388e30 P 1353649238cf4f7a909ccd8e1462cecd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } } peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } }
I20260925 07:26:07.684361 13816 leader_election.cc:288] T e29d07c10c374ade95f8c9d1ea388e30 P 1353649238cf4f7a909ccd8e1462cecd [CANDIDATE]: Term 1 election: Requested vote from peers 156cfffb10f04e838f2b80e19e59824a (127.5.158.66:36529), 9e74aea512cd41d1925f6b852e315c8f (127.5.158.67:41489)
I20260925 07:26:07.684541 13595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e29d07c10c374ade95f8c9d1ea388e30" candidate_uuid: "1353649238cf4f7a909ccd8e1462cecd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "156cfffb10f04e838f2b80e19e59824a"
I20260925 07:26:07.684541 13735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e29d07c10c374ade95f8c9d1ea388e30" candidate_uuid: "1353649238cf4f7a909ccd8e1462cecd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e74aea512cd41d1925f6b852e315c8f"
I20260925 07:26:07.684617 13735 raft_consensus.cc:3037] T e29d07c10c374ade95f8c9d1ea388e30 P 9e74aea512cd41d1925f6b852e315c8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:26:07.684617 13595 raft_consensus.cc:3037] T e29d07c10c374ade95f8c9d1ea388e30 P 156cfffb10f04e838f2b80e19e59824a [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:26:07.685359 13595 raft_consensus.cc:2454] T e29d07c10c374ade95f8c9d1ea388e30 P 156cfffb10f04e838f2b80e19e59824a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1353649238cf4f7a909ccd8e1462cecd in term 1.
I20260925 07:26:07.685359 13735 raft_consensus.cc:2454] T e29d07c10c374ade95f8c9d1ea388e30 P 9e74aea512cd41d1925f6b852e315c8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1353649238cf4f7a909ccd8e1462cecd in term 1.
I20260925 07:26:07.685667 13389 leader_election.cc:302] T e29d07c10c374ade95f8c9d1ea388e30 P 1353649238cf4f7a909ccd8e1462cecd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1353649238cf4f7a909ccd8e1462cecd, 9e74aea512cd41d1925f6b852e315c8f; no voters:
I20260925 07:26:07.685807 13816 raft_consensus.cc:2781] T e29d07c10c374ade95f8c9d1ea388e30 P 1353649238cf4f7a909ccd8e1462cecd [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:26:07.685873 13816 raft_consensus.cc:686] T e29d07c10c374ade95f8c9d1ea388e30 P 1353649238cf4f7a909ccd8e1462cecd [term 1 LEADER]: Becoming Leader. State: Replica: 1353649238cf4f7a909ccd8e1462cecd, State: Running, Role: LEADER
I20260925 07:26:07.685905 13816 consensus_queue.cc:237] T e29d07c10c374ade95f8c9d1ea388e30 P 1353649238cf4f7a909ccd8e1462cecd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } } peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } }
I20260925 07:26:07.686380 13298 catalog_manager.cc:5495] T e29d07c10c374ade95f8c9d1ea388e30 P 1353649238cf4f7a909ccd8e1462cecd reported cstate change: term changed from 0 to 1, leader changed from <none> to 1353649238cf4f7a909ccd8e1462cecd (127.5.158.65). New cstate: current_term: 1 leader_uuid: "1353649238cf4f7a909ccd8e1462cecd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } health_report { overall_health: UNKNOWN } } }
W20260925 07:26:07.707125 13782 tablet.cc:2369] T e29d07c10c374ade95f8c9d1ea388e30 P 9e74aea512cd41d1925f6b852e315c8f: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 25 07:26:07 dist-test-slave-1drq krb5kdc[13253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321166, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.158.126@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM
I20260925 07:26:07.743906 13840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:07.737617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:40461 (local address 127.0.0.1:58686)
0925 07:26:07.737796 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:07.737806 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:07.737890 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:07.738060 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:07.738060 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:07.738358 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:07.738395 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:07.738397 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.739058 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.739060 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:07.739570 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:07.739572 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.739953 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.739954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:07.740002 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:07.740455 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:07.740460 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:07.741816 (+ 1356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:26:07.743088 (+ 1272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:26:07.743094 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:26:07.743095 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:26:07.743273 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:26:07.743426 (+ 153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:26:07.743426 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:26:07.743426 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:26:07.743474 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:26:07.743654 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:26:07.743659 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:26:07.743696 (+ 37us) client_negotiation.cc:769] Sending connection context
0925 07:26:07.743742 (+ 46us) client_negotiation.cc:240] Negotiation successful
0925 07:26:07.743822 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":103,"threads_started":1}
I20260925 07:26:07.743968 13367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:07.737698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58686 (local address 127.5.158.126:40461)
0925 07:26:07.737772 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:07.737774 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:07.737871 (+ 97us) server_negotiation.cc:407] Connection header received
0925 07:26:07.737988 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:07.737989 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:07.737999 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:07.738051 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:07.738477 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.738986 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.739647 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.739881 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.739950 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:07.741899 (+ 1949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:26:07.741900 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:26:07.741900 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:26:07.741910 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:26:07.743025 (+ 1115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:26:07.743331 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:26:07.743331 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:26:07.743332 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:26:07.743375 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:26:07.743520 (+ 145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:26:07.743520 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:26:07.743520 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:26:07.743605 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:26:07.743649 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0925 07:26:07.743770 (+ 121us) server_negotiation.cc:299] Negotiation successful
0925 07:26:07.743862 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":49}
I20260925 07:26:07.745738 13298 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:58686:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260925 07:26:07.745945 13298 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 07:26:07.747368 13435 tablet_service.cc:1467] Processing CreateTablet for tablet 9281f84f8ee3427fbd168d4e3c23b51a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4995c64812bb4629ad60519bde0955f8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 07:26:07.747392 13575 tablet_service.cc:1467] Processing CreateTablet for tablet 9281f84f8ee3427fbd168d4e3c23b51a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4995c64812bb4629ad60519bde0955f8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 07:26:07.747483 13715 tablet_service.cc:1467] Processing CreateTablet for tablet 9281f84f8ee3427fbd168d4e3c23b51a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4995c64812bb4629ad60519bde0955f8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 07:26:07.747530 13435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9281f84f8ee3427fbd168d4e3c23b51a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:26:07.747550 13575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9281f84f8ee3427fbd168d4e3c23b51a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:26:07.747603 13715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9281f84f8ee3427fbd168d4e3c23b51a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:26:07.748941 13806 tablet_bootstrap.cc:492] T 9281f84f8ee3427fbd168d4e3c23b51a P 1353649238cf4f7a909ccd8e1462cecd: Bootstrap starting.
I20260925 07:26:07.748941 13808 tablet_bootstrap.cc:492] T 9281f84f8ee3427fbd168d4e3c23b51a P 9e74aea512cd41d1925f6b852e315c8f: Bootstrap starting.
I20260925 07:26:07.748991 13807 tablet_bootstrap.cc:492] T 9281f84f8ee3427fbd168d4e3c23b51a P 156cfffb10f04e838f2b80e19e59824a: Bootstrap starting.
I20260925 07:26:07.749338 13806 tablet_bootstrap.cc:654] T 9281f84f8ee3427fbd168d4e3c23b51a P 1353649238cf4f7a909ccd8e1462cecd: Neither blocks nor log segments found. Creating new log.
I20260925 07:26:07.749372 13807 tablet_bootstrap.cc:654] T 9281f84f8ee3427fbd168d4e3c23b51a P 156cfffb10f04e838f2b80e19e59824a: Neither blocks nor log segments found. Creating new log.
I20260925 07:26:07.749370 13808 tablet_bootstrap.cc:654] T 9281f84f8ee3427fbd168d4e3c23b51a P 9e74aea512cd41d1925f6b852e315c8f: Neither blocks nor log segments found. Creating new log.
I20260925 07:26:07.749892 13806 tablet_bootstrap.cc:492] T 9281f84f8ee3427fbd168d4e3c23b51a P 1353649238cf4f7a909ccd8e1462cecd: No bootstrap required, opened a new log
I20260925 07:26:07.749943 13806 ts_tablet_manager.cc:1397] T 9281f84f8ee3427fbd168d4e3c23b51a P 1353649238cf4f7a909ccd8e1462cecd: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 07:26:07.750061 13806 raft_consensus.cc:348] T 9281f84f8ee3427fbd168d4e3c23b51a P 1353649238cf4f7a909ccd8e1462cecd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } } peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } }
I20260925 07:26:07.750144 13806 raft_consensus.cc:374] T 9281f84f8ee3427fbd168d4e3c23b51a P 1353649238cf4f7a909ccd8e1462cecd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:26:07.750150 13808 tablet_bootstrap.cc:492] T 9281f84f8ee3427fbd168d4e3c23b51a P 9e74aea512cd41d1925f6b852e315c8f: No bootstrap required, opened a new log
I20260925 07:26:07.750168 13806 raft_consensus.cc:729] T 9281f84f8ee3427fbd168d4e3c23b51a P 1353649238cf4f7a909ccd8e1462cecd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1353649238cf4f7a909ccd8e1462cecd, State: Initialized, Role: FOLLOWER
I20260925 07:26:07.750186 13808 ts_tablet_manager.cc:1397] T 9281f84f8ee3427fbd168d4e3c23b51a P 9e74aea512cd41d1925f6b852e315c8f: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260925 07:26:07.750208 13806 consensus_queue.cc:260] T 9281f84f8ee3427fbd168d4e3c23b51a P 1353649238cf4f7a909ccd8e1462cecd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } } peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } }
I20260925 07:26:07.750308 13806 ts_tablet_manager.cc:1428] T 9281f84f8ee3427fbd168d4e3c23b51a P 1353649238cf4f7a909ccd8e1462cecd: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260925 07:26:07.750317 13808 raft_consensus.cc:348] T 9281f84f8ee3427fbd168d4e3c23b51a P 9e74aea512cd41d1925f6b852e315c8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } } peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } }
I20260925 07:26:07.750392 13808 raft_consensus.cc:374] T 9281f84f8ee3427fbd168d4e3c23b51a P 9e74aea512cd41d1925f6b852e315c8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:26:07.750422 13808 raft_consensus.cc:729] T 9281f84f8ee3427fbd168d4e3c23b51a P 9e74aea512cd41d1925f6b852e315c8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e74aea512cd41d1925f6b852e315c8f, State: Initialized, Role: FOLLOWER
I20260925 07:26:07.750396 13816 tablet_replica.cc:439] T 9281f84f8ee3427fbd168d4e3c23b51a P 1353649238cf4f7a909ccd8e1462cecd: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } } peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } } }
I20260925 07:26:07.750466 13816 tablet_replica.cc:442] T 9281f84f8ee3427fbd168d4e3c23b51a P 1353649238cf4f7a909ccd8e1462cecd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 07:26:07.750463 13808 consensus_queue.cc:260] T 9281f84f8ee3427fbd168d4e3c23b51a P 9e74aea512cd41d1925f6b852e315c8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } } peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } }
I20260925 07:26:07.750567 13808 ts_tablet_manager.cc:1428] T 9281f84f8ee3427fbd168d4e3c23b51a P 9e74aea512cd41d1925f6b852e315c8f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:07.750771 13807 tablet_bootstrap.cc:492] T 9281f84f8ee3427fbd168d4e3c23b51a P 156cfffb10f04e838f2b80e19e59824a: No bootstrap required, opened a new log
I20260925 07:26:07.750828 13807 ts_tablet_manager.cc:1397] T 9281f84f8ee3427fbd168d4e3c23b51a P 156cfffb10f04e838f2b80e19e59824a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 07:26:07.750844 13814 tablet_replica.cc:439] T 9281f84f8ee3427fbd168d4e3c23b51a P 9e74aea512cd41d1925f6b852e315c8f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } } peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } } }
I20260925 07:26:07.750923 13814 tablet_replica.cc:442] T 9281f84f8ee3427fbd168d4e3c23b51a P 9e74aea512cd41d1925f6b852e315c8f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 07:26:07.750972 13807 raft_consensus.cc:348] T 9281f84f8ee3427fbd168d4e3c23b51a P 156cfffb10f04e838f2b80e19e59824a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } } peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } }
I20260925 07:26:07.751040 13807 raft_consensus.cc:374] T 9281f84f8ee3427fbd168d4e3c23b51a P 156cfffb10f04e838f2b80e19e59824a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:26:07.751056 13807 raft_consensus.cc:729] T 9281f84f8ee3427fbd168d4e3c23b51a P 156cfffb10f04e838f2b80e19e59824a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 156cfffb10f04e838f2b80e19e59824a, State: Initialized, Role: FOLLOWER
I20260925 07:26:07.751096 13807 consensus_queue.cc:260] T 9281f84f8ee3427fbd168d4e3c23b51a P 156cfffb10f04e838f2b80e19e59824a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } } peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } }
I20260925 07:26:07.751183 13807 ts_tablet_manager.cc:1428] T 9281f84f8ee3427fbd168d4e3c23b51a P 156cfffb10f04e838f2b80e19e59824a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:07.751241 13813 tablet_replica.cc:439] T 9281f84f8ee3427fbd168d4e3c23b51a P 156cfffb10f04e838f2b80e19e59824a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } } peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } } }
I20260925 07:26:07.751327 13813 tablet_replica.cc:442] T 9281f84f8ee3427fbd168d4e3c23b51a P 156cfffb10f04e838f2b80e19e59824a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 07:26:07.807029 13814 raft_consensus.cc:482] T 9281f84f8ee3427fbd168d4e3c23b51a P 9e74aea512cd41d1925f6b852e315c8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 07:26:07.807112 13814 raft_consensus.cc:504] T 9281f84f8ee3427fbd168d4e3c23b51a P 9e74aea512cd41d1925f6b852e315c8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } } peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } }
I20260925 07:26:07.807502 13814 leader_election.cc:288] T 9281f84f8ee3427fbd168d4e3c23b51a P 9e74aea512cd41d1925f6b852e315c8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1353649238cf4f7a909ccd8e1462cecd (127.5.158.65:41851), 156cfffb10f04e838f2b80e19e59824a (127.5.158.66:36529)
I20260925 07:26:07.810542 13805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:07.807764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:33233 (local address 127.5.158.65:41851)
0925 07:26:07.807810 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:07.807812 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:07.807821 (+ 9us) server_negotiation.cc:407] Connection header received
0925 07:26:07.807849 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:07.807850 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:07.807863 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:07.807927 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:26:07.808110 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.808614 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.809591 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.810160 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.810213 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:07.810220 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 07:26:07.810469 (+ 249us) server_negotiation.cc:299] Negotiation successful
0925 07:26:07.810497 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 07:26:07.810556 13674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:07.807643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:41851 (local address 127.5.158.67:33233)
0925 07:26:07.807716 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:07.807731 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:07.807787 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:07.807957 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:07.807958 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:07.807960 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:26:07.808037 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:07.808039 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.808695 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.808696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:07.809507 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:07.809510 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.810294 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.810295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:07.810383 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:07.810387 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 07:26:07.810449 (+ 62us) client_negotiation.cc:240] Negotiation successful
0925 07:26:07.810487 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260925 07:26:07.810809 13455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9281f84f8ee3427fbd168d4e3c23b51a" candidate_uuid: "9e74aea512cd41d1925f6b852e315c8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1353649238cf4f7a909ccd8e1462cecd" is_pre_election: true
I20260925 07:26:07.811002 13455 raft_consensus.cc:2454] T 9281f84f8ee3427fbd168d4e3c23b51a P 1353649238cf4f7a909ccd8e1462cecd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e74aea512cd41d1925f6b852e315c8f in term 0.
I20260925 07:26:07.811158 13844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:07.807643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.66:36529 (local address 127.5.158.67:41217)
0925 07:26:07.807879 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:07.807888 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:07.807942 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:07.808088 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:07.808089 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:07.808090 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:26:07.808167 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:07.808169 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.808806 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.808807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:07.810024 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:07.810026 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.810878 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.810879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:07.810967 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:07.810970 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 07:26:07.811041 (+ 71us) client_negotiation.cc:240] Negotiation successful
0925 07:26:07.811082 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"mutex_wait_us":66,"thread_start_us":102,"threads_started":1}
I20260925 07:26:07.811185 13803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:07.807747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:41217 (local address 127.5.158.66:36529)
0925 07:26:07.807796 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:07.807798 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:07.807954 (+ 156us) server_negotiation.cc:407] Connection header received
0925 07:26:07.807987 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:07.807988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:07.807998 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:07.808063 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:26:07.808239 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.808723 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.810154 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.810793 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.810867 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:07.810876 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 07:26:07.811066 (+ 190us) server_negotiation.cc:299] Negotiation successful
0925 07:26:07.811105 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260925 07:26:07.811218 13669 leader_election.cc:302] T 9281f84f8ee3427fbd168d4e3c23b51a P 9e74aea512cd41d1925f6b852e315c8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1353649238cf4f7a909ccd8e1462cecd, 9e74aea512cd41d1925f6b852e315c8f; no voters:
I20260925 07:26:07.811383 13814 raft_consensus.cc:2781] T 9281f84f8ee3427fbd168d4e3c23b51a P 9e74aea512cd41d1925f6b852e315c8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 07:26:07.811436 13814 raft_consensus.cc:482] T 9281f84f8ee3427fbd168d4e3c23b51a P 9e74aea512cd41d1925f6b852e315c8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 07:26:07.811458 13814 raft_consensus.cc:3037] T 9281f84f8ee3427fbd168d4e3c23b51a P 9e74aea512cd41d1925f6b852e315c8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:26:07.811466 13595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9281f84f8ee3427fbd168d4e3c23b51a" candidate_uuid: "9e74aea512cd41d1925f6b852e315c8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "156cfffb10f04e838f2b80e19e59824a" is_pre_election: true
I20260925 07:26:07.811551 13595 raft_consensus.cc:2454] T 9281f84f8ee3427fbd168d4e3c23b51a P 156cfffb10f04e838f2b80e19e59824a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e74aea512cd41d1925f6b852e315c8f in term 0.
I20260925 07:26:07.812232 13814 raft_consensus.cc:504] T 9281f84f8ee3427fbd168d4e3c23b51a P 9e74aea512cd41d1925f6b852e315c8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } } peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } }
I20260925 07:26:07.812381 13814 leader_election.cc:288] T 9281f84f8ee3427fbd168d4e3c23b51a P 9e74aea512cd41d1925f6b852e315c8f [CANDIDATE]: Term 1 election: Requested vote from peers 1353649238cf4f7a909ccd8e1462cecd (127.5.158.65:41851), 156cfffb10f04e838f2b80e19e59824a (127.5.158.66:36529)
I20260925 07:26:07.812521 13455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9281f84f8ee3427fbd168d4e3c23b51a" candidate_uuid: "9e74aea512cd41d1925f6b852e315c8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1353649238cf4f7a909ccd8e1462cecd"
I20260925 07:26:07.812541 13595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9281f84f8ee3427fbd168d4e3c23b51a" candidate_uuid: "9e74aea512cd41d1925f6b852e315c8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "156cfffb10f04e838f2b80e19e59824a"
I20260925 07:26:07.812592 13455 raft_consensus.cc:3037] T 9281f84f8ee3427fbd168d4e3c23b51a P 1353649238cf4f7a909ccd8e1462cecd [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:26:07.812599 13595 raft_consensus.cc:3037] T 9281f84f8ee3427fbd168d4e3c23b51a P 156cfffb10f04e838f2b80e19e59824a [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:26:07.813290 13595 raft_consensus.cc:2454] T 9281f84f8ee3427fbd168d4e3c23b51a P 156cfffb10f04e838f2b80e19e59824a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e74aea512cd41d1925f6b852e315c8f in term 1.
I20260925 07:26:07.813467 13455 raft_consensus.cc:2454] T 9281f84f8ee3427fbd168d4e3c23b51a P 1353649238cf4f7a909ccd8e1462cecd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e74aea512cd41d1925f6b852e315c8f in term 1.
I20260925 07:26:07.813500 13668 leader_election.cc:302] T 9281f84f8ee3427fbd168d4e3c23b51a P 9e74aea512cd41d1925f6b852e315c8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 156cfffb10f04e838f2b80e19e59824a, 9e74aea512cd41d1925f6b852e315c8f; no voters:
I20260925 07:26:07.813565 13814 raft_consensus.cc:2781] T 9281f84f8ee3427fbd168d4e3c23b51a P 9e74aea512cd41d1925f6b852e315c8f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:26:07.813700 13814 raft_consensus.cc:686] T 9281f84f8ee3427fbd168d4e3c23b51a P 9e74aea512cd41d1925f6b852e315c8f [term 1 LEADER]: Becoming Leader. State: Replica: 9e74aea512cd41d1925f6b852e315c8f, State: Running, Role: LEADER
I20260925 07:26:07.813752 13814 consensus_queue.cc:237] T 9281f84f8ee3427fbd168d4e3c23b51a P 9e74aea512cd41d1925f6b852e315c8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } } peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } }
I20260925 07:26:07.814077 13845 tablet_replica.cc:439] T 9281f84f8ee3427fbd168d4e3c23b51a P 9e74aea512cd41d1925f6b852e315c8f: TxnStatusTablet state changed. Reason: New leader 9e74aea512cd41d1925f6b852e315c8f. Latest consensus state: current_term: 1 leader_uuid: "9e74aea512cd41d1925f6b852e315c8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } } peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } } }
I20260925 07:26:07.814169 13845 tablet_replica.cc:442] T 9281f84f8ee3427fbd168d4e3c23b51a P 9e74aea512cd41d1925f6b852e315c8f: This TxnStatusTablet replica's current role is: LEADER
W20260925 07:26:07.814265 13642 tablet.cc:2369] T 9281f84f8ee3427fbd168d4e3c23b51a P 156cfffb10f04e838f2b80e19e59824a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 07:26:07.814410 13298 catalog_manager.cc:5495] T 9281f84f8ee3427fbd168d4e3c23b51a P 9e74aea512cd41d1925f6b852e315c8f reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e74aea512cd41d1925f6b852e315c8f (127.5.158.67). New cstate: current_term: 1 leader_uuid: "9e74aea512cd41d1925f6b852e315c8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } health_report { overall_health: HEALTHY } } }
I20260925 07:26:07.814561 13847 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260925 07:26:07.814656 13847 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260925 07:26:07.825134 13802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:07.822501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32878 (local address 127.5.158.67:41489)
0925 07:26:07.822543 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:07.822545 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:07.822553 (+ 8us) server_negotiation.cc:407] Connection header received
0925 07:26:07.822624 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:07.822625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:07.822639 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:07.822706 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 07:26:07.822864 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.823567 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.824361 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.824662 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.824751 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:07.824870 (+ 119us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 07:26:07.824962 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 07:26:07.825018 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 07:26:07.825050 (+ 32us) server_negotiation.cc:299] Negotiation successful
0925 07:26:07.825065 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 07:26:07.825165 13840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:07.822418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.67:41489 (local address 127.0.0.1:32878)
0925 07:26:07.822486 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:07.822498 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:07.822545 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:07.822715 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:07.822716 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:07.822717 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 07:26:07.822768 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:07.822770 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.823658 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.823659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:07.824279 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:07.824282 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.824755 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.824756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:07.824804 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:07.824815 (+ 11us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 07:26:07.825010 (+ 195us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 07:26:07.825015 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 07:26:07.825042 (+ 27us) client_negotiation.cc:240] Negotiation successful
0925 07:26:07.825072 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260925 07:26:07.825973 13595 raft_consensus.cc:1261] T 9281f84f8ee3427fbd168d4e3c23b51a P 156cfffb10f04e838f2b80e19e59824a [term 1 FOLLOWER]: Refusing update from remote peer 9e74aea512cd41d1925f6b852e315c8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:26:07.826030 13455 raft_consensus.cc:1261] T 9281f84f8ee3427fbd168d4e3c23b51a P 1353649238cf4f7a909ccd8e1462cecd [term 1 FOLLOWER]: Refusing update from remote peer 9e74aea512cd41d1925f6b852e315c8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:26:07.826228 13845 consensus_queue.cc:1035] T 9281f84f8ee3427fbd168d4e3c23b51a P 9e74aea512cd41d1925f6b852e315c8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:26:07.826329 13814 consensus_queue.cc:1035] T 9281f84f8ee3427fbd168d4e3c23b51a P 9e74aea512cd41d1925f6b852e315c8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:26:07.826776 13813 tablet_replica.cc:439] T 9281f84f8ee3427fbd168d4e3c23b51a P 156cfffb10f04e838f2b80e19e59824a: TxnStatusTablet state changed. Reason: New leader 9e74aea512cd41d1925f6b852e315c8f. Latest consensus state: current_term: 1 leader_uuid: "9e74aea512cd41d1925f6b852e315c8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } } peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } } }
I20260925 07:26:07.826860 13813 tablet_replica.cc:442] T 9281f84f8ee3427fbd168d4e3c23b51a P 156cfffb10f04e838f2b80e19e59824a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 07:26:07.826848 13816 tablet_replica.cc:439] T 9281f84f8ee3427fbd168d4e3c23b51a P 1353649238cf4f7a909ccd8e1462cecd: TxnStatusTablet state changed. Reason: New leader 9e74aea512cd41d1925f6b852e315c8f. Latest consensus state: current_term: 1 leader_uuid: "9e74aea512cd41d1925f6b852e315c8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } } peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } } }
I20260925 07:26:07.826922 13816 tablet_replica.cc:442] T 9281f84f8ee3427fbd168d4e3c23b51a P 1353649238cf4f7a909ccd8e1462cecd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 07:26:07.827689 13845 tablet_replica.cc:439] T 9281f84f8ee3427fbd168d4e3c23b51a P 9e74aea512cd41d1925f6b852e315c8f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9e74aea512cd41d1925f6b852e315c8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } } peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } } }
I20260925 07:26:07.827801 13845 tablet_replica.cc:442] T 9281f84f8ee3427fbd168d4e3c23b51a P 9e74aea512cd41d1925f6b852e315c8f: This TxnStatusTablet replica's current role is: LEADER
I20260925 07:26:07.828047 13816 tablet_replica.cc:439] T 9281f84f8ee3427fbd168d4e3c23b51a P 1353649238cf4f7a909ccd8e1462cecd: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9e74aea512cd41d1925f6b852e315c8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } } peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } } }
I20260925 07:26:07.828115 13816 tablet_replica.cc:442] T 9281f84f8ee3427fbd168d4e3c23b51a P 1353649238cf4f7a909ccd8e1462cecd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 07:26:07.828934 13813 tablet_replica.cc:439] T 9281f84f8ee3427fbd168d4e3c23b51a P 156cfffb10f04e838f2b80e19e59824a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9e74aea512cd41d1925f6b852e315c8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } } peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } } }
I20260925 07:26:07.829016 13813 tablet_replica.cc:442] T 9281f84f8ee3427fbd168d4e3c23b51a P 156cfffb10f04e838f2b80e19e59824a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 07:26:07.829219 13845 tablet_replica.cc:439] T 9281f84f8ee3427fbd168d4e3c23b51a P 9e74aea512cd41d1925f6b852e315c8f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9e74aea512cd41d1925f6b852e315c8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } } peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } } }
I20260925 07:26:07.829285 13845 tablet_replica.cc:442] T 9281f84f8ee3427fbd168d4e3c23b51a P 9e74aea512cd41d1925f6b852e315c8f: This TxnStatusTablet replica's current role is: LEADER
I20260925 07:26:07.832866 13802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:07.830760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:54987 (local address 127.5.158.67:41489)
0925 07:26:07.830795 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:07.830796 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:07.830869 (+ 73us) server_negotiation.cc:407] Connection header received
0925 07:26:07.830951 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:07.830952 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:07.830960 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:07.831008 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 07:26:07.831092 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.831492 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.832230 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.832411 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.832482 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:07.832666 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 07:26:07.832693 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 07:26:07.832748 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 07:26:07.832802 (+ 54us) server_negotiation.cc:299] Negotiation successful
0925 07:26:07.832815 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 07:26:07.832907 13854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:07.830703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.67:41489 (local address 127.5.158.65:54987)
0925 07:26:07.830819 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:07.830827 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:07.830892 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:07.831012 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:07.831012 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:07.831013 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 07:26:07.831051 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:07.831052 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.831575 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.831576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:07.832115 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:07.832117 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.832488 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.832489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:07.832546 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:07.832564 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 07:26:07.832748 (+ 184us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 07:26:07.832751 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 07:26:07.832798 (+ 47us) client_negotiation.cc:240] Negotiation successful
0925 07:26:07.832835 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":66,"threads_started":1}
I20260925 07:26:07.834654 13735 raft_consensus.cc:1261] T e29d07c10c374ade95f8c9d1ea388e30 P 9e74aea512cd41d1925f6b852e315c8f [term 1 FOLLOWER]: Refusing update from remote peer 1353649238cf4f7a909ccd8e1462cecd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:26:07.834743 13595 raft_consensus.cc:1261] T e29d07c10c374ade95f8c9d1ea388e30 P 156cfffb10f04e838f2b80e19e59824a [term 1 FOLLOWER]: Refusing update from remote peer 1353649238cf4f7a909ccd8e1462cecd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:26:07.834864 13812 consensus_queue.cc:1035] T e29d07c10c374ade95f8c9d1ea388e30 P 1353649238cf4f7a909ccd8e1462cecd [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:26:07.835078 13812 consensus_queue.cc:1035] T e29d07c10c374ade95f8c9d1ea388e30 P 1353649238cf4f7a909ccd8e1462cecd [LEADER]: Connected to new peer: Peer: permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:26:07.837824 13820 mvcc.cc:205] Tried to move back new op lower bound from 7333155503449300992 to 7333155502841761792. Current Snapshot: MvccSnapshot[applied={T|T < 7333155503449300992}]
I20260925 07:26:07.842976 13805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:07.840573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:39063 (local address 127.5.158.65:41851)
0925 07:26:07.840606 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:07.840608 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:07.840633 (+ 25us) server_negotiation.cc:407] Connection header received
0925 07:26:07.840698 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:07.840698 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:07.840705 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:07.840745 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 07:26:07.840881 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.841359 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.842406 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.842567 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.842615 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:07.842813 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 07:26:07.842835 (+ 22us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 07:26:07.842869 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0925 07:26:07.842924 (+ 55us) server_negotiation.cc:299] Negotiation successful
0925 07:26:07.842937 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 07:26:07.843029 13787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:07.840520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:41851 (local address 127.5.158.67:39063)
0925 07:26:07.840582 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:07.840594 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:07.840645 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:07.840767 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:07.840767 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:07.840769 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 07:26:07.840822 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:07.840823 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.841441 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.841442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:07.842323 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:07.842326 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.842645 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.842646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:07.842728 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:07.842740 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 07:26:07.842873 (+ 133us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 07:26:07.842875 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 07:26:07.842923 (+ 48us) client_negotiation.cc:240] Negotiation successful
0925 07:26:07.842960 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260925 07:26:07.850608 13297 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58676:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 07:26:07.850701 13297 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58676:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 07:26:07.851274 13297 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f5e596c5f3284f62a2c097cde2041a96: Sending DeleteTablet for 3 replicas of tablet e29d07c10c374ade95f8c9d1ea388e30
I20260925 07:26:07.851477 13435 tablet_service.cc:1514] Processing DeleteTablet for tablet e29d07c10c374ade95f8c9d1ea388e30 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:26:07 UTC) from {username='oryx', principal='oryx/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:44124
I20260925 07:26:07.851495 13715 tablet_service.cc:1514] Processing DeleteTablet for tablet e29d07c10c374ade95f8c9d1ea388e30 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:26:07 UTC) from {username='oryx', principal='oryx/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:32862
I20260925 07:26:07.851505 13575 tablet_service.cc:1514] Processing DeleteTablet for tablet e29d07c10c374ade95f8c9d1ea388e30 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:26:07 UTC) from {username='oryx', principal='oryx/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:48152
I20260925 07:26:07.851572 13830 tablet_replica.cc:330] T e29d07c10c374ade95f8c9d1ea388e30 P 1353649238cf4f7a909ccd8e1462cecd: stopping tablet replica
I20260925 07:26:07.851570 13829 tablet_replica.cc:330] T e29d07c10c374ade95f8c9d1ea388e30 P 9e74aea512cd41d1925f6b852e315c8f: stopping tablet replica
I20260925 07:26:07.851614 13829 raft_consensus.cc:2229] T e29d07c10c374ade95f8c9d1ea388e30 P 9e74aea512cd41d1925f6b852e315c8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 07:26:07.851619 13830 raft_consensus.cc:2229] T e29d07c10c374ade95f8c9d1ea388e30 P 1353649238cf4f7a909ccd8e1462cecd [term 1 LEADER]: Raft consensus shutting down.
I20260925 07:26:07.851660 13830 raft_consensus.cc:2258] T e29d07c10c374ade95f8c9d1ea388e30 P 1353649238cf4f7a909ccd8e1462cecd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:26:07.851673 13829 raft_consensus.cc:2258] T e29d07c10c374ade95f8c9d1ea388e30 P 9e74aea512cd41d1925f6b852e315c8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:26:07.851716 13828 tablet_replica.cc:330] T e29d07c10c374ade95f8c9d1ea388e30 P 156cfffb10f04e838f2b80e19e59824a: stopping tablet replica
I20260925 07:26:07.851769 13828 raft_consensus.cc:2229] T e29d07c10c374ade95f8c9d1ea388e30 P 156cfffb10f04e838f2b80e19e59824a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 07:26:07.851800 13828 raft_consensus.cc:2258] T e29d07c10c374ade95f8c9d1ea388e30 P 156cfffb10f04e838f2b80e19e59824a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:26:07.851941 13830 ts_tablet_manager.cc:1905] T e29d07c10c374ade95f8c9d1ea388e30 P 1353649238cf4f7a909ccd8e1462cecd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:26:07.851941 13829 ts_tablet_manager.cc:1905] T e29d07c10c374ade95f8c9d1ea388e30 P 9e74aea512cd41d1925f6b852e315c8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:26:07.853543 13830 ts_tablet_manager.cc:1918] T e29d07c10c374ade95f8c9d1ea388e30 P 1353649238cf4f7a909ccd8e1462cecd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 07:26:07.853598 13830 log.cc:1196] T e29d07c10c374ade95f8c9d1ea388e30 P 1353649238cf4f7a909ccd8e1462cecd: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-0/wal/wals/e29d07c10c374ade95f8c9d1ea388e30
I20260925 07:26:07.853598 13829 ts_tablet_manager.cc:1918] T e29d07c10c374ade95f8c9d1ea388e30 P 9e74aea512cd41d1925f6b852e315c8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 07:26:07.853646 13829 log.cc:1196] T e29d07c10c374ade95f8c9d1ea388e30 P 9e74aea512cd41d1925f6b852e315c8f: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-2/wal/wals/e29d07c10c374ade95f8c9d1ea388e30
I20260925 07:26:07.853809 13828 ts_tablet_manager.cc:1905] T e29d07c10c374ade95f8c9d1ea388e30 P 156cfffb10f04e838f2b80e19e59824a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:26:07.853833 13830 ts_tablet_manager.cc:1939] T e29d07c10c374ade95f8c9d1ea388e30 P 1353649238cf4f7a909ccd8e1462cecd: Deleting consensus metadata
I20260925 07:26:07.853885 13829 ts_tablet_manager.cc:1939] T e29d07c10c374ade95f8c9d1ea388e30 P 9e74aea512cd41d1925f6b852e315c8f: Deleting consensus metadata
I20260925 07:26:07.854305 13285 catalog_manager.cc:4841] TS 1353649238cf4f7a909ccd8e1462cecd (127.5.158.65:41851): tablet e29d07c10c374ade95f8c9d1ea388e30 (table test-table [id=bf752289d9d945609cba519cf234d148]) successfully deleted
I20260925 07:26:07.854379 13285 catalog_manager.cc:4841] TS 9e74aea512cd41d1925f6b852e315c8f (127.5.158.67:41489): tablet e29d07c10c374ade95f8c9d1ea388e30 (table test-table [id=bf752289d9d945609cba519cf234d148]) successfully deleted
I20260925 07:26:07.855298 13828 ts_tablet_manager.cc:1918] T e29d07c10c374ade95f8c9d1ea388e30 P 156cfffb10f04e838f2b80e19e59824a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 07:26:07.855372 13828 log.cc:1196] T e29d07c10c374ade95f8c9d1ea388e30 P 156cfffb10f04e838f2b80e19e59824a: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-1/wal/wals/e29d07c10c374ade95f8c9d1ea388e30
I20260925 07:26:07.855556 13828 ts_tablet_manager.cc:1939] T e29d07c10c374ade95f8c9d1ea388e30 P 156cfffb10f04e838f2b80e19e59824a: Deleting consensus metadata
I20260925 07:26:07.855808 13284 catalog_manager.cc:4841] TS 156cfffb10f04e838f2b80e19e59824a (127.5.158.66:36529): tablet e29d07c10c374ade95f8c9d1ea388e30 (table test-table [id=bf752289d9d945609cba519cf234d148]) successfully deleted
Sep 25 07:26:07 dist-test-slave-1drq krb5kdc[13253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 07:26:07 dist-test-slave-1drq krb5kdc[13253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 07:26:07 dist-test-slave-1drq krb5kdc[13253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
W20260925 07:26:07.878386 13367 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:26:07.873126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58702 (local address 127.5.158.126:40461)
0925 07:26:07.873173 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:07.873175 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:07.873309 (+ 134us) server_negotiation.cc:407] Connection header received
0925 07:26:07.873340 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:07.873341 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:07.873351 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:07.873423 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:07.873939 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.874637 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.875303 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.875555 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.875624 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:07.878329 (+ 2705us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58702: BlockingRecv error: recv got EOF from 127.0.0.1:58702 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Sep 25 07:26:07 dist-test-slave-1drq krb5kdc[13253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM
I20260925 07:26:07.885842 13367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:07.879124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58712 (local address 127.5.158.126:40461)
0925 07:26:07.879154 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:07.879156 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:07.879242 (+ 86us) server_negotiation.cc:407] Connection header received
0925 07:26:07.879294 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:07.879295 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:07.879303 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:07.879359 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:07.879806 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.880484 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.881054 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.881362 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.881427 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:07.883354 (+ 1927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:26:07.883355 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:26:07.883355 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:26:07.883362 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:26:07.884819 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:26:07.885160 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:26:07.885161 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:26:07.885161 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:26:07.885217 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:26:07.885381 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:26:07.885382 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:26:07.885382 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:26:07.885498 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:26:07.885585 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0925 07:26:07.885684 (+ 99us) server_negotiation.cc:299] Negotiation successful
0925 07:26:07.885754 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260925 07:26:07.887245 13297 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58712:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 07:26:07.887414 13297 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 07:26:07.888708 13435 tablet_service.cc:1467] Processing CreateTablet for tablet 918b5a54813c440d9c9e616cff8434fc (DEFAULT_TABLE table=test-table [id=7a8dc65e9d5b44debd5f28018ab2d688]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:26:07.888775 13575 tablet_service.cc:1467] Processing CreateTablet for tablet 918b5a54813c440d9c9e616cff8434fc (DEFAULT_TABLE table=test-table [id=7a8dc65e9d5b44debd5f28018ab2d688]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:26:07.888795 13715 tablet_service.cc:1467] Processing CreateTablet for tablet 918b5a54813c440d9c9e616cff8434fc (DEFAULT_TABLE table=test-table [id=7a8dc65e9d5b44debd5f28018ab2d688]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:26:07.888833 13435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 918b5a54813c440d9c9e616cff8434fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:26:07.888873 13575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 918b5a54813c440d9c9e616cff8434fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:26:07.888906 13715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 918b5a54813c440d9c9e616cff8434fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:26:07.890172 13806 tablet_bootstrap.cc:492] T 918b5a54813c440d9c9e616cff8434fc P 1353649238cf4f7a909ccd8e1462cecd: Bootstrap starting.
I20260925 07:26:07.890273 13807 tablet_bootstrap.cc:492] T 918b5a54813c440d9c9e616cff8434fc P 156cfffb10f04e838f2b80e19e59824a: Bootstrap starting.
I20260925 07:26:07.890511 13806 tablet_bootstrap.cc:654] T 918b5a54813c440d9c9e616cff8434fc P 1353649238cf4f7a909ccd8e1462cecd: Neither blocks nor log segments found. Creating new log.
I20260925 07:26:07.890681 13807 tablet_bootstrap.cc:654] T 918b5a54813c440d9c9e616cff8434fc P 156cfffb10f04e838f2b80e19e59824a: Neither blocks nor log segments found. Creating new log.
I20260925 07:26:07.891000 13808 tablet_bootstrap.cc:492] T 918b5a54813c440d9c9e616cff8434fc P 9e74aea512cd41d1925f6b852e315c8f: Bootstrap starting.
I20260925 07:26:07.891362 13806 tablet_bootstrap.cc:492] T 918b5a54813c440d9c9e616cff8434fc P 1353649238cf4f7a909ccd8e1462cecd: No bootstrap required, opened a new log
I20260925 07:26:07.891387 13808 tablet_bootstrap.cc:654] T 918b5a54813c440d9c9e616cff8434fc P 9e74aea512cd41d1925f6b852e315c8f: Neither blocks nor log segments found. Creating new log.
I20260925 07:26:07.891427 13806 ts_tablet_manager.cc:1397] T 918b5a54813c440d9c9e616cff8434fc P 1353649238cf4f7a909ccd8e1462cecd: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 07:26:07.891541 13806 raft_consensus.cc:348] T 918b5a54813c440d9c9e616cff8434fc P 1353649238cf4f7a909ccd8e1462cecd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } } peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } }
I20260925 07:26:07.891634 13806 raft_consensus.cc:374] T 918b5a54813c440d9c9e616cff8434fc P 1353649238cf4f7a909ccd8e1462cecd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:26:07.891680 13806 raft_consensus.cc:729] T 918b5a54813c440d9c9e616cff8434fc P 1353649238cf4f7a909ccd8e1462cecd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1353649238cf4f7a909ccd8e1462cecd, State: Initialized, Role: FOLLOWER
I20260925 07:26:07.891721 13806 consensus_queue.cc:260] T 918b5a54813c440d9c9e616cff8434fc P 1353649238cf4f7a909ccd8e1462cecd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } } peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } }
I20260925 07:26:07.891826 13806 ts_tablet_manager.cc:1428] T 918b5a54813c440d9c9e616cff8434fc P 1353649238cf4f7a909ccd8e1462cecd: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260925 07:26:07.891940 13807 tablet_bootstrap.cc:492] T 918b5a54813c440d9c9e616cff8434fc P 156cfffb10f04e838f2b80e19e59824a: No bootstrap required, opened a new log
I20260925 07:26:07.892015 13807 ts_tablet_manager.cc:1397] T 918b5a54813c440d9c9e616cff8434fc P 156cfffb10f04e838f2b80e19e59824a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 07:26:07.892148 13807 raft_consensus.cc:348] T 918b5a54813c440d9c9e616cff8434fc P 156cfffb10f04e838f2b80e19e59824a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } } peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } }
I20260925 07:26:07.892241 13807 raft_consensus.cc:374] T 918b5a54813c440d9c9e616cff8434fc P 156cfffb10f04e838f2b80e19e59824a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:26:07.892269 13807 raft_consensus.cc:729] T 918b5a54813c440d9c9e616cff8434fc P 156cfffb10f04e838f2b80e19e59824a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 156cfffb10f04e838f2b80e19e59824a, State: Initialized, Role: FOLLOWER
I20260925 07:26:07.892308 13807 consensus_queue.cc:260] T 918b5a54813c440d9c9e616cff8434fc P 156cfffb10f04e838f2b80e19e59824a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } } peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } }
I20260925 07:26:07.892388 13807 ts_tablet_manager.cc:1428] T 918b5a54813c440d9c9e616cff8434fc P 156cfffb10f04e838f2b80e19e59824a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:07.892741 13808 tablet_bootstrap.cc:492] T 918b5a54813c440d9c9e616cff8434fc P 9e74aea512cd41d1925f6b852e315c8f: No bootstrap required, opened a new log
I20260925 07:26:07.892789 13808 ts_tablet_manager.cc:1397] T 918b5a54813c440d9c9e616cff8434fc P 9e74aea512cd41d1925f6b852e315c8f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260925 07:26:07.892870 13808 raft_consensus.cc:348] T 918b5a54813c440d9c9e616cff8434fc P 9e74aea512cd41d1925f6b852e315c8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } } peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } }
I20260925 07:26:07.892930 13808 raft_consensus.cc:374] T 918b5a54813c440d9c9e616cff8434fc P 9e74aea512cd41d1925f6b852e315c8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:26:07.892948 13808 raft_consensus.cc:729] T 918b5a54813c440d9c9e616cff8434fc P 9e74aea512cd41d1925f6b852e315c8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e74aea512cd41d1925f6b852e315c8f, State: Initialized, Role: FOLLOWER
I20260925 07:26:07.892983 13808 consensus_queue.cc:260] T 918b5a54813c440d9c9e616cff8434fc P 9e74aea512cd41d1925f6b852e315c8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } } peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } }
I20260925 07:26:07.893060 13808 ts_tablet_manager.cc:1428] T 918b5a54813c440d9c9e616cff8434fc P 9e74aea512cd41d1925f6b852e315c8f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:07.925057 13813 raft_consensus.cc:482] T 918b5a54813c440d9c9e616cff8434fc P 156cfffb10f04e838f2b80e19e59824a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 07:26:07.925158 13813 raft_consensus.cc:504] T 918b5a54813c440d9c9e616cff8434fc P 156cfffb10f04e838f2b80e19e59824a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } } peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } }
I20260925 07:26:07.925426 13813 leader_election.cc:288] T 918b5a54813c440d9c9e616cff8434fc P 156cfffb10f04e838f2b80e19e59824a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1353649238cf4f7a909ccd8e1462cecd (127.5.158.65:41851), 9e74aea512cd41d1925f6b852e315c8f (127.5.158.67:41489)
I20260925 07:26:07.928467 13805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:07.925692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:37461 (local address 127.5.158.65:41851)
0925 07:26:07.925739 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:07.925742 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:07.925809 (+ 67us) server_negotiation.cc:407] Connection header received
0925 07:26:07.925932 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:07.925933 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:07.925948 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:07.926006 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:26:07.926192 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.926830 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.927798 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.928135 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.928229 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:07.928237 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 07:26:07.928377 (+ 140us) server_negotiation.cc:299] Negotiation successful
0925 07:26:07.928410 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 07:26:07.928494 13878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:07.925640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:41851 (local address 127.5.158.66:37461)
0925 07:26:07.925746 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:07.925757 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:07.925838 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:07.926025 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:07.926026 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:07.926028 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:26:07.926101 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:07.926103 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.926910 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.926911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:07.927692 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:07.927694 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.928229 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.928230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:07.928300 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:07.928305 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 07:26:07.928371 (+ 66us) client_negotiation.cc:240] Negotiation successful
0925 07:26:07.928409 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"mutex_wait_us":1,"thread_start_us":49,"threads_started":1}
I20260925 07:26:07.928763 13455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "918b5a54813c440d9c9e616cff8434fc" candidate_uuid: "156cfffb10f04e838f2b80e19e59824a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1353649238cf4f7a909ccd8e1462cecd" is_pre_election: true
I20260925 07:26:07.928860 13455 raft_consensus.cc:2454] T 918b5a54813c440d9c9e616cff8434fc P 1353649238cf4f7a909ccd8e1462cecd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 156cfffb10f04e838f2b80e19e59824a in term 0.
I20260925 07:26:07.929024 13529 leader_election.cc:302] T 918b5a54813c440d9c9e616cff8434fc P 156cfffb10f04e838f2b80e19e59824a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1353649238cf4f7a909ccd8e1462cecd, 156cfffb10f04e838f2b80e19e59824a; no voters:
I20260925 07:26:07.929162 13813 raft_consensus.cc:2781] T 918b5a54813c440d9c9e616cff8434fc P 156cfffb10f04e838f2b80e19e59824a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 07:26:07.929196 13813 raft_consensus.cc:482] T 918b5a54813c440d9c9e616cff8434fc P 156cfffb10f04e838f2b80e19e59824a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 07:26:07.929220 13813 raft_consensus.cc:3037] T 918b5a54813c440d9c9e616cff8434fc P 156cfffb10f04e838f2b80e19e59824a [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:26:07.929571 13802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:07.925836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:49059 (local address 127.5.158.67:41489)
0925 07:26:07.925874 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:07.925876 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:07.925925 (+ 49us) server_negotiation.cc:407] Connection header received
0925 07:26:07.926001 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:07.926002 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:07.926011 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:07.926056 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:26:07.926579 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.927290 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.928674 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.929216 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.929261 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:07.929268 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 07:26:07.929462 (+ 194us) server_negotiation.cc:299] Negotiation successful
0925 07:26:07.929499 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 07:26:07.929572 13879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:07.925788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.67:41489 (local address 127.5.158.66:49059)
0925 07:26:07.925874 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:07.925882 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:07.925947 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:07.926072 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:07.926073 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:07.926076 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:26:07.926463 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:07.926466 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.927390 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.927390 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:07.928569 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:07.928572 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.929323 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.929324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:07.929390 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:07.929393 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 07:26:07.929441 (+ 48us) client_negotiation.cc:240] Negotiation successful
0925 07:26:07.929477 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75,"thread_start_us":45,"threads_started":1}
I20260925 07:26:07.930049 13813 raft_consensus.cc:504] T 918b5a54813c440d9c9e616cff8434fc P 156cfffb10f04e838f2b80e19e59824a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } } peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } }
I20260925 07:26:07.930167 13813 leader_election.cc:288] T 918b5a54813c440d9c9e616cff8434fc P 156cfffb10f04e838f2b80e19e59824a [CANDIDATE]: Term 1 election: Requested vote from peers 1353649238cf4f7a909ccd8e1462cecd (127.5.158.65:41851), 9e74aea512cd41d1925f6b852e315c8f (127.5.158.67:41489)
I20260925 07:26:07.930358 13455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "918b5a54813c440d9c9e616cff8434fc" candidate_uuid: "156cfffb10f04e838f2b80e19e59824a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1353649238cf4f7a909ccd8e1462cecd"
I20260925 07:26:07.930429 13455 raft_consensus.cc:3037] T 918b5a54813c440d9c9e616cff8434fc P 1353649238cf4f7a909ccd8e1462cecd [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:26:07.930406 13735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "918b5a54813c440d9c9e616cff8434fc" candidate_uuid: "156cfffb10f04e838f2b80e19e59824a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e74aea512cd41d1925f6b852e315c8f" is_pre_election: true
I20260925 07:26:07.930471 13735 raft_consensus.cc:2454] T 918b5a54813c440d9c9e616cff8434fc P 9e74aea512cd41d1925f6b852e315c8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 156cfffb10f04e838f2b80e19e59824a in term 0.
I20260925 07:26:07.930452 13734 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "918b5a54813c440d9c9e616cff8434fc" candidate_uuid: "156cfffb10f04e838f2b80e19e59824a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e74aea512cd41d1925f6b852e315c8f"
I20260925 07:26:07.930642 13734 raft_consensus.cc:3037] T 918b5a54813c440d9c9e616cff8434fc P 9e74aea512cd41d1925f6b852e315c8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:26:07.931154 13455 raft_consensus.cc:2454] T 918b5a54813c440d9c9e616cff8434fc P 1353649238cf4f7a909ccd8e1462cecd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 156cfffb10f04e838f2b80e19e59824a in term 1.
I20260925 07:26:07.931319 13529 leader_election.cc:302] T 918b5a54813c440d9c9e616cff8434fc P 156cfffb10f04e838f2b80e19e59824a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1353649238cf4f7a909ccd8e1462cecd, 156cfffb10f04e838f2b80e19e59824a; no voters:
I20260925 07:26:07.931337 13734 raft_consensus.cc:2454] T 918b5a54813c440d9c9e616cff8434fc P 9e74aea512cd41d1925f6b852e315c8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 156cfffb10f04e838f2b80e19e59824a in term 1.
I20260925 07:26:07.931444 13813 raft_consensus.cc:2781] T 918b5a54813c440d9c9e616cff8434fc P 156cfffb10f04e838f2b80e19e59824a [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:26:07.931568 13813 raft_consensus.cc:686] T 918b5a54813c440d9c9e616cff8434fc P 156cfffb10f04e838f2b80e19e59824a [term 1 LEADER]: Becoming Leader. State: Replica: 156cfffb10f04e838f2b80e19e59824a, State: Running, Role: LEADER
I20260925 07:26:07.931624 13813 consensus_queue.cc:237] T 918b5a54813c440d9c9e616cff8434fc P 156cfffb10f04e838f2b80e19e59824a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } } peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } }
I20260925 07:26:07.932331 13297 catalog_manager.cc:5495] T 918b5a54813c440d9c9e616cff8434fc P 156cfffb10f04e838f2b80e19e59824a reported cstate change: term changed from 0 to 1, leader changed from <none> to 156cfffb10f04e838f2b80e19e59824a (127.5.158.66). New cstate: current_term: 1 leader_uuid: "156cfffb10f04e838f2b80e19e59824a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } health_report { overall_health: UNKNOWN } } }
I20260925 07:26:07.940410 13803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:07.937891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48156 (local address 127.5.158.66:36529)
0925 07:26:07.937953 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:07.937955 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:07.937962 (+ 7us) server_negotiation.cc:407] Connection header received
0925 07:26:07.937989 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:07.937989 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:07.938006 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:07.938087 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 07:26:07.938203 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.938695 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.939575 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:07.939836 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:07.939906 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:07.940075 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 07:26:07.940174 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 07:26:07.940246 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 07:26:07.940292 (+ 46us) server_negotiation.cc:299] Negotiation successful
0925 07:26:07.940310 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 07:26:07.941313 13455 raft_consensus.cc:1261] T 918b5a54813c440d9c9e616cff8434fc P 1353649238cf4f7a909ccd8e1462cecd [term 1 FOLLOWER]: Refusing update from remote peer 156cfffb10f04e838f2b80e19e59824a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:26:07.941385 13734 raft_consensus.cc:1261] T 918b5a54813c440d9c9e616cff8434fc P 9e74aea512cd41d1925f6b852e315c8f [term 1 FOLLOWER]: Refusing update from remote peer 156cfffb10f04e838f2b80e19e59824a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:26:07.941577 13813 consensus_queue.cc:1035] T 918b5a54813c440d9c9e616cff8434fc P 156cfffb10f04e838f2b80e19e59824a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:26:07.941684 13880 consensus_queue.cc:1035] T 918b5a54813c440d9c9e616cff8434fc P 156cfffb10f04e838f2b80e19e59824a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:26:07.943635 13821 mvcc.cc:205] Tried to move back new op lower bound from 7333155503885877248 to 7333155503848554496. Current Snapshot: MvccSnapshot[applied={T|T < 7333155503885877248}]
I20260925 07:26:07.945163 13296 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58712:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 07:26:07.945245 13296 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58712:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 07:26:07.945779 13296 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f5e596c5f3284f62a2c097cde2041a96: Sending DeleteTablet for 3 replicas of tablet 918b5a54813c440d9c9e616cff8434fc
I20260925 07:26:07.946012 13435 tablet_service.cc:1514] Processing DeleteTablet for tablet 918b5a54813c440d9c9e616cff8434fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:26:07 UTC) from {username='oryx', principal='oryx/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:44124
I20260925 07:26:07.946089 13830 tablet_replica.cc:330] T 918b5a54813c440d9c9e616cff8434fc P 1353649238cf4f7a909ccd8e1462cecd: stopping tablet replica
I20260925 07:26:07.946125 13830 raft_consensus.cc:2229] T 918b5a54813c440d9c9e616cff8434fc P 1353649238cf4f7a909ccd8e1462cecd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 07:26:07.946138 13575 tablet_service.cc:1514] Processing DeleteTablet for tablet 918b5a54813c440d9c9e616cff8434fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:26:07 UTC) from {username='oryx', principal='oryx/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:48152
I20260925 07:26:07.946163 13830 raft_consensus.cc:2258] T 918b5a54813c440d9c9e616cff8434fc P 1353649238cf4f7a909ccd8e1462cecd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:26:07.946208 13828 tablet_replica.cc:330] T 918b5a54813c440d9c9e616cff8434fc P 156cfffb10f04e838f2b80e19e59824a: stopping tablet replica
I20260925 07:26:07.946246 13828 raft_consensus.cc:2229] T 918b5a54813c440d9c9e616cff8434fc P 156cfffb10f04e838f2b80e19e59824a [term 1 LEADER]: Raft consensus shutting down.
I20260925 07:26:07.946312 13828 raft_consensus.cc:2258] T 918b5a54813c440d9c9e616cff8434fc P 156cfffb10f04e838f2b80e19e59824a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:26:07.946645 13715 tablet_service.cc:1514] Processing DeleteTablet for tablet 918b5a54813c440d9c9e616cff8434fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:26:07 UTC) from {username='oryx', principal='oryx/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:32862
I20260925 07:26:07.946657 13830 ts_tablet_manager.cc:1905] T 918b5a54813c440d9c9e616cff8434fc P 1353649238cf4f7a909ccd8e1462cecd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:26:07.946657 13828 ts_tablet_manager.cc:1905] T 918b5a54813c440d9c9e616cff8434fc P 156cfffb10f04e838f2b80e19e59824a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:26:07.946724 13829 tablet_replica.cc:330] T 918b5a54813c440d9c9e616cff8434fc P 9e74aea512cd41d1925f6b852e315c8f: stopping tablet replica
I20260925 07:26:07.946769 13829 raft_consensus.cc:2229] T 918b5a54813c440d9c9e616cff8434fc P 9e74aea512cd41d1925f6b852e315c8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 07:26:07.946805 13829 raft_consensus.cc:2258] T 918b5a54813c440d9c9e616cff8434fc P 9e74aea512cd41d1925f6b852e315c8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:26:07.946949 13296 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58712:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 07:26:07.947079 13296 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 07:26:07.948038 13830 ts_tablet_manager.cc:1918] T 918b5a54813c440d9c9e616cff8434fc P 1353649238cf4f7a909ccd8e1462cecd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 07:26:07.948088 13830 log.cc:1196] T 918b5a54813c440d9c9e616cff8434fc P 1353649238cf4f7a909ccd8e1462cecd: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-0/wal/wals/918b5a54813c440d9c9e616cff8434fc
I20260925 07:26:07.948122 13828 ts_tablet_manager.cc:1918] T 918b5a54813c440d9c9e616cff8434fc P 156cfffb10f04e838f2b80e19e59824a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 07:26:07.948164 13828 log.cc:1196] T 918b5a54813c440d9c9e616cff8434fc P 156cfffb10f04e838f2b80e19e59824a: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-1/wal/wals/918b5a54813c440d9c9e616cff8434fc
I20260925 07:26:07.948298 13830 ts_tablet_manager.cc:1939] T 918b5a54813c440d9c9e616cff8434fc P 1353649238cf4f7a909ccd8e1462cecd: Deleting consensus metadata
I20260925 07:26:07.948339 13828 ts_tablet_manager.cc:1939] T 918b5a54813c440d9c9e616cff8434fc P 156cfffb10f04e838f2b80e19e59824a: Deleting consensus metadata
I20260925 07:26:07.948325 13435 tablet_service.cc:1467] Processing CreateTablet for tablet 4385aae82bc040339de3418897280aa8 (DEFAULT_TABLE table=test-table [id=45a6cca05d8648608707fd9d579286f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:26:07.948428 13435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4385aae82bc040339de3418897280aa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:26:07.948563 13285 catalog_manager.cc:4841] TS 1353649238cf4f7a909ccd8e1462cecd (127.5.158.65:41851): tablet 918b5a54813c440d9c9e616cff8434fc (table test-table [id=7a8dc65e9d5b44debd5f28018ab2d688]) successfully deleted
I20260925 07:26:07.949535 13806 tablet_bootstrap.cc:492] T 4385aae82bc040339de3418897280aa8 P 1353649238cf4f7a909ccd8e1462cecd: Bootstrap starting.
I20260925 07:26:07.949676 13284 catalog_manager.cc:4841] TS 156cfffb10f04e838f2b80e19e59824a (127.5.158.66:36529): tablet 918b5a54813c440d9c9e616cff8434fc (table test-table [id=7a8dc65e9d5b44debd5f28018ab2d688]) successfully deleted
I20260925 07:26:07.949810 13806 tablet_bootstrap.cc:654] T 4385aae82bc040339de3418897280aa8 P 1353649238cf4f7a909ccd8e1462cecd: Neither blocks nor log segments found. Creating new log.
I20260925 07:26:07.949815 13715 tablet_service.cc:1467] Processing CreateTablet for tablet 4385aae82bc040339de3418897280aa8 (DEFAULT_TABLE table=test-table [id=45a6cca05d8648608707fd9d579286f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:26:07.949900 13715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4385aae82bc040339de3418897280aa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:26:07.950366 13806 tablet_bootstrap.cc:492] T 4385aae82bc040339de3418897280aa8 P 1353649238cf4f7a909ccd8e1462cecd: No bootstrap required, opened a new log
I20260925 07:26:07.950407 13806 ts_tablet_manager.cc:1397] T 4385aae82bc040339de3418897280aa8 P 1353649238cf4f7a909ccd8e1462cecd: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 07:26:07.950512 13806 raft_consensus.cc:348] T 4385aae82bc040339de3418897280aa8 P 1353649238cf4f7a909ccd8e1462cecd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } } peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } }
I20260925 07:26:07.950565 13806 raft_consensus.cc:374] T 4385aae82bc040339de3418897280aa8 P 1353649238cf4f7a909ccd8e1462cecd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:26:07.950584 13806 raft_consensus.cc:729] T 4385aae82bc040339de3418897280aa8 P 1353649238cf4f7a909ccd8e1462cecd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1353649238cf4f7a909ccd8e1462cecd, State: Initialized, Role: FOLLOWER
I20260925 07:26:07.950630 13806 consensus_queue.cc:260] T 4385aae82bc040339de3418897280aa8 P 1353649238cf4f7a909ccd8e1462cecd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } } peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } }
I20260925 07:26:07.950706 13806 ts_tablet_manager.cc:1428] T 4385aae82bc040339de3418897280aa8 P 1353649238cf4f7a909ccd8e1462cecd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:07.951045 13808 tablet_bootstrap.cc:492] T 4385aae82bc040339de3418897280aa8 P 9e74aea512cd41d1925f6b852e315c8f: Bootstrap starting.
I20260925 07:26:07.951661 13808 tablet_bootstrap.cc:654] T 4385aae82bc040339de3418897280aa8 P 9e74aea512cd41d1925f6b852e315c8f: Neither blocks nor log segments found. Creating new log.
I20260925 07:26:07.952056 13575 tablet_service.cc:1467] Processing CreateTablet for tablet 4385aae82bc040339de3418897280aa8 (DEFAULT_TABLE table=test-table [id=45a6cca05d8648608707fd9d579286f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:26:07.952155 13575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4385aae82bc040339de3418897280aa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:26:07.953264 13807 tablet_bootstrap.cc:492] T 4385aae82bc040339de3418897280aa8 P 156cfffb10f04e838f2b80e19e59824a: Bootstrap starting.
I20260925 07:26:07.953621 13807 tablet_bootstrap.cc:654] T 4385aae82bc040339de3418897280aa8 P 156cfffb10f04e838f2b80e19e59824a: Neither blocks nor log segments found. Creating new log.
I20260925 07:26:07.954246 13807 tablet_bootstrap.cc:492] T 4385aae82bc040339de3418897280aa8 P 156cfffb10f04e838f2b80e19e59824a: No bootstrap required, opened a new log
I20260925 07:26:07.954291 13807 ts_tablet_manager.cc:1397] T 4385aae82bc040339de3418897280aa8 P 156cfffb10f04e838f2b80e19e59824a: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260925 07:26:07.954417 13807 raft_consensus.cc:348] T 4385aae82bc040339de3418897280aa8 P 156cfffb10f04e838f2b80e19e59824a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } } peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } }
I20260925 07:26:07.954470 13807 raft_consensus.cc:374] T 4385aae82bc040339de3418897280aa8 P 156cfffb10f04e838f2b80e19e59824a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:26:07.954486 13807 raft_consensus.cc:729] T 4385aae82bc040339de3418897280aa8 P 156cfffb10f04e838f2b80e19e59824a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 156cfffb10f04e838f2b80e19e59824a, State: Initialized, Role: FOLLOWER
I20260925 07:26:07.954520 13807 consensus_queue.cc:260] T 4385aae82bc040339de3418897280aa8 P 156cfffb10f04e838f2b80e19e59824a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } } peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } }
I20260925 07:26:07.954595 13807 ts_tablet_manager.cc:1428] T 4385aae82bc040339de3418897280aa8 P 156cfffb10f04e838f2b80e19e59824a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:07.955003 13808 tablet_bootstrap.cc:492] T 4385aae82bc040339de3418897280aa8 P 9e74aea512cd41d1925f6b852e315c8f: No bootstrap required, opened a new log
I20260925 07:26:07.955046 13808 ts_tablet_manager.cc:1397] T 4385aae82bc040339de3418897280aa8 P 9e74aea512cd41d1925f6b852e315c8f: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260925 07:26:07.955169 13808 raft_consensus.cc:348] T 4385aae82bc040339de3418897280aa8 P 9e74aea512cd41d1925f6b852e315c8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } } peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } }
I20260925 07:26:07.955232 13808 raft_consensus.cc:374] T 4385aae82bc040339de3418897280aa8 P 9e74aea512cd41d1925f6b852e315c8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:26:07.955248 13808 raft_consensus.cc:729] T 4385aae82bc040339de3418897280aa8 P 9e74aea512cd41d1925f6b852e315c8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e74aea512cd41d1925f6b852e315c8f, State: Initialized, Role: FOLLOWER
I20260925 07:26:07.955281 13808 consensus_queue.cc:260] T 4385aae82bc040339de3418897280aa8 P 9e74aea512cd41d1925f6b852e315c8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } } peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } }
I20260925 07:26:07.955351 13808 ts_tablet_manager.cc:1428] T 4385aae82bc040339de3418897280aa8 P 9e74aea512cd41d1925f6b852e315c8f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:07.955813 13829 ts_tablet_manager.cc:1905] T 918b5a54813c440d9c9e616cff8434fc P 9e74aea512cd41d1925f6b852e315c8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:26:07.957050 13829 ts_tablet_manager.cc:1918] T 918b5a54813c440d9c9e616cff8434fc P 9e74aea512cd41d1925f6b852e315c8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 07:26:07.957142 13829 log.cc:1196] T 918b5a54813c440d9c9e616cff8434fc P 9e74aea512cd41d1925f6b852e315c8f: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-2/wal/wals/918b5a54813c440d9c9e616cff8434fc
I20260925 07:26:07.957373 13829 ts_tablet_manager.cc:1939] T 918b5a54813c440d9c9e616cff8434fc P 9e74aea512cd41d1925f6b852e315c8f: Deleting consensus metadata
I20260925 07:26:07.957662 13285 catalog_manager.cc:4841] TS 9e74aea512cd41d1925f6b852e315c8f (127.5.158.67:41489): tablet 918b5a54813c440d9c9e616cff8434fc (table test-table [id=7a8dc65e9d5b44debd5f28018ab2d688]) successfully deleted
I20260925 07:26:08.055094 13880 raft_consensus.cc:482] T 4385aae82bc040339de3418897280aa8 P 156cfffb10f04e838f2b80e19e59824a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 07:26:08.055187 13880 raft_consensus.cc:504] T 4385aae82bc040339de3418897280aa8 P 156cfffb10f04e838f2b80e19e59824a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } } peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } }
I20260925 07:26:08.055380 13880 leader_election.cc:288] T 4385aae82bc040339de3418897280aa8 P 156cfffb10f04e838f2b80e19e59824a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1353649238cf4f7a909ccd8e1462cecd (127.5.158.65:41851), 9e74aea512cd41d1925f6b852e315c8f (127.5.158.67:41489)
I20260925 07:26:08.055611 13455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4385aae82bc040339de3418897280aa8" candidate_uuid: "156cfffb10f04e838f2b80e19e59824a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1353649238cf4f7a909ccd8e1462cecd" is_pre_election: true
I20260925 07:26:08.055694 13455 raft_consensus.cc:2454] T 4385aae82bc040339de3418897280aa8 P 1353649238cf4f7a909ccd8e1462cecd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 156cfffb10f04e838f2b80e19e59824a in term 0.
I20260925 07:26:08.055603 13734 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4385aae82bc040339de3418897280aa8" candidate_uuid: "156cfffb10f04e838f2b80e19e59824a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e74aea512cd41d1925f6b852e315c8f" is_pre_election: true
I20260925 07:26:08.055738 13734 raft_consensus.cc:2454] T 4385aae82bc040339de3418897280aa8 P 9e74aea512cd41d1925f6b852e315c8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 156cfffb10f04e838f2b80e19e59824a in term 0.
I20260925 07:26:08.055845 13529 leader_election.cc:302] T 4385aae82bc040339de3418897280aa8 P 156cfffb10f04e838f2b80e19e59824a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1353649238cf4f7a909ccd8e1462cecd, 156cfffb10f04e838f2b80e19e59824a; no voters:
I20260925 07:26:08.055971 13880 raft_consensus.cc:2781] T 4385aae82bc040339de3418897280aa8 P 156cfffb10f04e838f2b80e19e59824a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 07:26:08.056022 13880 raft_consensus.cc:482] T 4385aae82bc040339de3418897280aa8 P 156cfffb10f04e838f2b80e19e59824a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 07:26:08.056039 13880 raft_consensus.cc:3037] T 4385aae82bc040339de3418897280aa8 P 156cfffb10f04e838f2b80e19e59824a [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:26:08.056716 13880 raft_consensus.cc:504] T 4385aae82bc040339de3418897280aa8 P 156cfffb10f04e838f2b80e19e59824a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } } peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } }
I20260925 07:26:08.056849 13880 leader_election.cc:288] T 4385aae82bc040339de3418897280aa8 P 156cfffb10f04e838f2b80e19e59824a [CANDIDATE]: Term 1 election: Requested vote from peers 1353649238cf4f7a909ccd8e1462cecd (127.5.158.65:41851), 9e74aea512cd41d1925f6b852e315c8f (127.5.158.67:41489)
I20260925 07:26:08.057042 13455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4385aae82bc040339de3418897280aa8" candidate_uuid: "156cfffb10f04e838f2b80e19e59824a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1353649238cf4f7a909ccd8e1462cecd"
I20260925 07:26:08.057068 13734 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4385aae82bc040339de3418897280aa8" candidate_uuid: "156cfffb10f04e838f2b80e19e59824a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e74aea512cd41d1925f6b852e315c8f"
I20260925 07:26:08.057152 13455 raft_consensus.cc:3037] T 4385aae82bc040339de3418897280aa8 P 1353649238cf4f7a909ccd8e1462cecd [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:26:08.057165 13734 raft_consensus.cc:3037] T 4385aae82bc040339de3418897280aa8 P 9e74aea512cd41d1925f6b852e315c8f [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:26:08.057958 13455 raft_consensus.cc:2454] T 4385aae82bc040339de3418897280aa8 P 1353649238cf4f7a909ccd8e1462cecd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 156cfffb10f04e838f2b80e19e59824a in term 1.
I20260925 07:26:08.057960 13734 raft_consensus.cc:2454] T 4385aae82bc040339de3418897280aa8 P 9e74aea512cd41d1925f6b852e315c8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 156cfffb10f04e838f2b80e19e59824a in term 1.
I20260925 07:26:08.058172 13529 leader_election.cc:302] T 4385aae82bc040339de3418897280aa8 P 156cfffb10f04e838f2b80e19e59824a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1353649238cf4f7a909ccd8e1462cecd, 156cfffb10f04e838f2b80e19e59824a; no voters:
I20260925 07:26:08.058276 13880 raft_consensus.cc:2781] T 4385aae82bc040339de3418897280aa8 P 156cfffb10f04e838f2b80e19e59824a [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:26:08.058338 13880 raft_consensus.cc:686] T 4385aae82bc040339de3418897280aa8 P 156cfffb10f04e838f2b80e19e59824a [term 1 LEADER]: Becoming Leader. State: Replica: 156cfffb10f04e838f2b80e19e59824a, State: Running, Role: LEADER
I20260925 07:26:08.058419 13880 consensus_queue.cc:237] T 4385aae82bc040339de3418897280aa8 P 156cfffb10f04e838f2b80e19e59824a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } } peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } }
I20260925 07:26:08.059090 13296 catalog_manager.cc:5495] T 4385aae82bc040339de3418897280aa8 P 156cfffb10f04e838f2b80e19e59824a reported cstate change: term changed from 0 to 1, leader changed from <none> to 156cfffb10f04e838f2b80e19e59824a (127.5.158.66). New cstate: current_term: 1 leader_uuid: "156cfffb10f04e838f2b80e19e59824a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "156cfffb10f04e838f2b80e19e59824a" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 36529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 } health_report { overall_health: UNKNOWN } } }
I20260925 07:26:08.076911 13802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:08.074768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:51369 (local address 127.5.158.67:41489)
0925 07:26:08.074807 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:08.074809 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:08.074913 (+ 104us) server_negotiation.cc:407] Connection header received
0925 07:26:08.074966 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:08.074966 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:08.074976 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:08.075039 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 07:26:08.075180 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:08.075669 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:08.076372 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:08.076562 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:08.076609 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:08.076739 (+ 130us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 07:26:08.076762 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 07:26:08.076799 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0925 07:26:08.076845 (+ 46us) server_negotiation.cc:299] Negotiation successful
0925 07:26:08.076865 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 07:26:08.076937 13892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:08.074705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.67:41489 (local address 127.5.158.66:51369)
0925 07:26:08.074844 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:08.074855 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:08.074925 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:08.075039 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:08.075040 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:08.075041 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 07:26:08.075079 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:08.075080 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:08.075729 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:08.075730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:08.076302 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:08.076304 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:08.076621 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:08.076622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:08.076660 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:08.076673 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 07:26:08.076809 (+ 136us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 07:26:08.076811 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 07:26:08.076844 (+ 33us) client_negotiation.cc:240] Negotiation successful
0925 07:26:08.076877 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":92,"threads_started":1}
I20260925 07:26:08.078516 13734 raft_consensus.cc:1261] T 4385aae82bc040339de3418897280aa8 P 9e74aea512cd41d1925f6b852e315c8f [term 1 FOLLOWER]: Refusing update from remote peer 156cfffb10f04e838f2b80e19e59824a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:26:08.078665 13455 raft_consensus.cc:1261] T 4385aae82bc040339de3418897280aa8 P 1353649238cf4f7a909ccd8e1462cecd [term 1 FOLLOWER]: Refusing update from remote peer 156cfffb10f04e838f2b80e19e59824a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:26:08.078718 13813 consensus_queue.cc:1035] T 4385aae82bc040339de3418897280aa8 P 156cfffb10f04e838f2b80e19e59824a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e74aea512cd41d1925f6b852e315c8f" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:26:08.078841 13880 consensus_queue.cc:1035] T 4385aae82bc040339de3418897280aa8 P 156cfffb10f04e838f2b80e19e59824a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1353649238cf4f7a909ccd8e1462cecd" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:26:08.084750 13803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:08.082830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:45113 (local address 127.5.158.66:36529)
0925 07:26:08.082869 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:08.082870 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:08.082876 (+ 6us) server_negotiation.cc:407] Connection header received
0925 07:26:08.082890 (+ 14us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:08.082891 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:08.082897 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:08.082930 (+ 33us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 07:26:08.083056 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:08.083473 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:08.084146 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:08.084323 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:08.084367 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:08.084530 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 07:26:08.084560 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 07:26:08.084607 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 07:26:08.084667 (+ 60us) server_negotiation.cc:299] Negotiation successful
0925 07:26:08.084688 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 07:26:08.084780 13787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:08.082706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.66:36529 (local address 127.5.158.67:45113)
0925 07:26:08.082735 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:08.082744 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:08.082775 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:08.082946 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:08.082947 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:08.082948 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 07:26:08.082984 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:08.082985 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:08.083531 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:08.083532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:08.084064 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:08.084065 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:08.084381 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:08.084382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:08.084415 (+ 33us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:08.084444 (+ 29us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 07:26:08.084620 (+ 176us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 07:26:08.084621 (+ 1us) client_negotiation.cc:769] Sending connection context
0925 07:26:08.084663 (+ 42us) client_negotiation.cc:240] Negotiation successful
0925 07:26:08.084709 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14}
I20260925 07:26:08.090487 13296 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58712:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 07:26:08.090554 13296 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58712:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 07:26:08.090999 13296 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f5e596c5f3284f62a2c097cde2041a96: Sending DeleteTablet for 3 replicas of tablet 4385aae82bc040339de3418897280aa8
I20260925 07:26:08.091221 13575 tablet_service.cc:1514] Processing DeleteTablet for tablet 4385aae82bc040339de3418897280aa8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:26:08 UTC) from {username='oryx', principal='oryx/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:48152
I20260925 07:26:08.091236 13435 tablet_service.cc:1514] Processing DeleteTablet for tablet 4385aae82bc040339de3418897280aa8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:26:08 UTC) from {username='oryx', principal='oryx/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:44124
I20260925 07:26:08.091301 13828 tablet_replica.cc:330] T 4385aae82bc040339de3418897280aa8 P 156cfffb10f04e838f2b80e19e59824a: stopping tablet replica
I20260925 07:26:08.091315 13830 tablet_replica.cc:330] T 4385aae82bc040339de3418897280aa8 P 1353649238cf4f7a909ccd8e1462cecd: stopping tablet replica
I20260925 07:26:08.091353 13830 raft_consensus.cc:2229] T 4385aae82bc040339de3418897280aa8 P 1353649238cf4f7a909ccd8e1462cecd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 07:26:08.091352 13828 raft_consensus.cc:2229] T 4385aae82bc040339de3418897280aa8 P 156cfffb10f04e838f2b80e19e59824a [term 1 LEADER]: Raft consensus shutting down.
I20260925 07:26:08.091380 13830 raft_consensus.cc:2258] T 4385aae82bc040339de3418897280aa8 P 1353649238cf4f7a909ccd8e1462cecd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:26:08.091403 13828 raft_consensus.cc:2258] T 4385aae82bc040339de3418897280aa8 P 156cfffb10f04e838f2b80e19e59824a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:26:08.091501 13715 tablet_service.cc:1514] Processing DeleteTablet for tablet 4385aae82bc040339de3418897280aa8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:26:08 UTC) from {username='oryx', principal='oryx/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:32862
I20260925 07:26:08.091531 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 13370
I20260925 07:26:08.091607 13829 tablet_replica.cc:330] T 4385aae82bc040339de3418897280aa8 P 9e74aea512cd41d1925f6b852e315c8f: stopping tablet replica
I20260925 07:26:08.091639 13830 ts_tablet_manager.cc:1905] T 4385aae82bc040339de3418897280aa8 P 1353649238cf4f7a909ccd8e1462cecd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:26:08.091660 13829 raft_consensus.cc:2229] T 4385aae82bc040339de3418897280aa8 P 9e74aea512cd41d1925f6b852e315c8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 07:26:08.091691 13829 raft_consensus.cc:2258] T 4385aae82bc040339de3418897280aa8 P 9e74aea512cd41d1925f6b852e315c8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:26:08.093667 13829 ts_tablet_manager.cc:1905] T 4385aae82bc040339de3418897280aa8 P 9e74aea512cd41d1925f6b852e315c8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:26:08.095329 13829 ts_tablet_manager.cc:1918] T 4385aae82bc040339de3418897280aa8 P 9e74aea512cd41d1925f6b852e315c8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 07:26:08.095327 13828 ts_tablet_manager.cc:1905] T 4385aae82bc040339de3418897280aa8 P 156cfffb10f04e838f2b80e19e59824a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:26:08.095402 13829 log.cc:1196] T 4385aae82bc040339de3418897280aa8 P 9e74aea512cd41d1925f6b852e315c8f: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-2/wal/wals/4385aae82bc040339de3418897280aa8
I20260925 07:26:08.095593 13829 ts_tablet_manager.cc:1939] T 4385aae82bc040339de3418897280aa8 P 9e74aea512cd41d1925f6b852e315c8f: Deleting consensus metadata
I20260925 07:26:08.095894 13285 catalog_manager.cc:4841] TS 9e74aea512cd41d1925f6b852e315c8f (127.5.158.67:41489): tablet 4385aae82bc040339de3418897280aa8 (table test-table [id=45a6cca05d8648608707fd9d579286f6]) successfully deleted
I20260925 07:26:08.096885 13828 ts_tablet_manager.cc:1918] T 4385aae82bc040339de3418897280aa8 P 156cfffb10f04e838f2b80e19e59824a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 07:26:08.096951 13828 log.cc:1196] T 4385aae82bc040339de3418897280aa8 P 156cfffb10f04e838f2b80e19e59824a: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790321110323796-5753-0/minicluster-data/ts-1/wal/wals/4385aae82bc040339de3418897280aa8
I20260925 07:26:08.097190 13828 ts_tablet_manager.cc:1939] T 4385aae82bc040339de3418897280aa8 P 156cfffb10f04e838f2b80e19e59824a: Deleting consensus metadata
I20260925 07:26:08.097453 13284 catalog_manager.cc:4841] TS 156cfffb10f04e838f2b80e19e59824a (127.5.158.66:36529): tablet 4385aae82bc040339de3418897280aa8 (table test-table [id=45a6cca05d8648608707fd9d579286f6]) successfully deleted
W20260925 07:26:08.098467 13285 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.158.65:41851 (error 108)
I20260925 07:26:08.098663 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 13510
I20260925 07:26:08.099200 13897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:08.098715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:41851 (local address 127.0.0.1:44138)
0925 07:26:08.099127 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:08.099161 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.65:41851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":45,"threads_started":1}
W20260925 07:26:08.099344 13285 catalog_manager.cc:4584] TS 1353649238cf4f7a909ccd8e1462cecd (127.5.158.65:41851): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4385aae82bc040339de3418897280aa8: Network error: Client connection negotiation failed: client connection to 127.5.158.65:41851: connect: Connection refused (error 111)
I20260925 07:26:08.104387 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 13650
I20260925 07:26:08.109740 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 13266
2026-09-25T07:26:08Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2292 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790321110323796-5753-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 07:26:08 dist-test-slave-1drq krb5kdc[13900](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 07:26:08 dist-test-slave-1drq krb5kdc[13900](info): set up 2 sockets
Sep 25 07:26:08 dist-test-slave-1drq krb5kdc[13900](info): commencing operation
krb5kdc: starting...
W20260925 07:26:09.069351 5753 mini_kdc.cc:121] Time spent starting KDC: real 0.940s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Sep 25 07:26:09 dist-test-slave-1drq krb5kdc[13900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321169, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T07:26:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T07:26:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
I20260925 07:26:09.175370 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:35641
--webserver_interface=127.5.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:41273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.158.126:35641
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:26:09.248339 13913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:26:09.248543 13913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:26:09.248565 13913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:26:09.249691 13913 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:26:09.249728 13913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:26:09.249742 13913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:26:09.249752 13913 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:26:09.249802 13913 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:26:09.251121 13913 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:41273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790321110323796-5753-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.158.126:35641
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:35641
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13913
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:26:09.251351 13913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:26:09.251586 13913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:26:09.256029 13919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:26:09.256002 13918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:26:09.256043 13913 server_base.cc:1034] running on GCE node
W20260925 07:26:09.256016 13921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:26:09.256531 13913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:26:09.256815 13913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:26:09.257987 13913 hybrid_clock.cc:648] HybridClock initialized: now 1790321169258023 us; error 96 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-25T07:26:09Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1152 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 07:26:09 dist-test-slave-1drq krb5kdc[13926](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 07:26:09 dist-test-slave-1drq krb5kdc[13926](info): set up 2 sockets
Sep 25 07:26:09 dist-test-slave-1drq krb5kdc[13926](info): commencing operation
krb5kdc: starting...
W20260925 07:26:10.257205 5753 mini_kdc.cc:121] Time spent starting KDC: real 0.975s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Sep 25 07:26:10 dist-test-slave-1drq krb5kdc[13926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321170, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T07:26:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T07:26:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
I20260925 07:26:10.362080 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:42669
--webserver_interface=127.5.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:41139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.158.126:42669
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:26:10.439035 13939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:26:10.439256 13939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:26:10.439291 13939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:26:10.440407 13939 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:26:10.440471 13939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:26:10.440497 13939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:26:10.440527 13939 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:26:10.440551 13939 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:26:10.441998 13939 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:41139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.158.126:42669
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:42669
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13939
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:26:10.442274 13939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:26:10.442487 13939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:26:10.447144 13945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:26:10.447144 13944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:26:10.447168 13947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:26:10.447252 13939 server_base.cc:1034] running on GCE node
I20260925 07:26:10.447599 13939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:26:10.447827 13939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:26:10.448962 13939 hybrid_clock.cc:648] HybridClock initialized: now 1790321170448951 us; error 28 us; skew 500 ppm
Sep 25 07:26:10 dist-test-slave-1drq krb5kdc[13926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321170, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:26:10.454912 13939 init.cc:377] Logged in from keytab as kudu/127.5.158.126@KRBTEST.COM (short username kudu)
I20260925 07:26:10.455320 13939 webserver.cc:460] Webserver started at http://127.5.158.126:34529/ using document root <none> and password file <none>
I20260925 07:26:10.455528 13939 fs_manager.cc:359] Metadata directory not provided
I20260925 07:26:10.455587 13939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:26:10.455721 13939 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:26:10.456678 13939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/master-0/data/instance:
uuid: "054bbea9422f4472bb2b431b5088181c"
format_stamp: "Formatted at 2026-09-25 07:26:10 on dist-test-slave-1drq"
server_key: "2eaa64c83bdf8e0f0efbd21e2f4efc8d"
server_key_iv: "0881ec14a58ce5131b39b9cb9defa2d0"
server_key_version: "encryptionkey@0"
I20260925 07:26:10.457024 13939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/master-0/wal/instance:
uuid: "054bbea9422f4472bb2b431b5088181c"
format_stamp: "Formatted at 2026-09-25 07:26:10 on dist-test-slave-1drq"
server_key: "2eaa64c83bdf8e0f0efbd21e2f4efc8d"
server_key_iv: "0881ec14a58ce5131b39b9cb9defa2d0"
server_key_version: "encryptionkey@0"
I20260925 07:26:10.458293 13939 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 07:26:10.458953 13954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:10.459106 13939 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 07:26:10.459194 13939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "054bbea9422f4472bb2b431b5088181c"
format_stamp: "Formatted at 2026-09-25 07:26:10 on dist-test-slave-1drq"
server_key: "2eaa64c83bdf8e0f0efbd21e2f4efc8d"
server_key_iv: "0881ec14a58ce5131b39b9cb9defa2d0"
server_key_version: "encryptionkey@0"
I20260925 07:26:10.459267 13939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:26:10.488921 13939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:26:10.491549 13939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:26:10.491690 13939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:26:10.496192 13939 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.126:42669
I20260925 07:26:10.496255 14006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.126:42669 every 8 connection(s)
I20260925 07:26:10.496595 13939 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:26:10.497671 14007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:26:10.501294 14007 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 054bbea9422f4472bb2b431b5088181c: Bootstrap starting.
I20260925 07:26:10.502015 14007 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 054bbea9422f4472bb2b431b5088181c: Neither blocks nor log segments found. Creating new log.
I20260925 07:26:10.502341 14007 log.cc:824] T 00000000000000000000000000000000 P 054bbea9422f4472bb2b431b5088181c: Log is configured to *not* fsync() on all Append() calls
I20260925 07:26:10.503204 14007 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 054bbea9422f4472bb2b431b5088181c: No bootstrap required, opened a new log
I20260925 07:26:10.505940 14007 raft_consensus.cc:348] T 00000000000000000000000000000000 P 054bbea9422f4472bb2b431b5088181c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "054bbea9422f4472bb2b431b5088181c" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 42669 } }
I20260925 07:26:10.506134 14007 raft_consensus.cc:374] T 00000000000000000000000000000000 P 054bbea9422f4472bb2b431b5088181c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:26:10.506218 14007 raft_consensus.cc:729] T 00000000000000000000000000000000 P 054bbea9422f4472bb2b431b5088181c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 054bbea9422f4472bb2b431b5088181c, State: Initialized, Role: FOLLOWER
I20260925 07:26:10.506381 14007 consensus_queue.cc:260] T 00000000000000000000000000000000 P 054bbea9422f4472bb2b431b5088181c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "054bbea9422f4472bb2b431b5088181c" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 42669 } }
I20260925 07:26:10.506493 14007 raft_consensus.cc:388] T 00000000000000000000000000000000 P 054bbea9422f4472bb2b431b5088181c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 07:26:10.506589 14007 raft_consensus.cc:482] T 00000000000000000000000000000000 P 054bbea9422f4472bb2b431b5088181c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 07:26:10.506651 14007 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 054bbea9422f4472bb2b431b5088181c [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:26:10.506644 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 13939
I20260925 07:26:10.506747 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/master-0/wal/instance
I20260925 07:26:10.506933 5753 external_mini_cluster.cc:1428] Setting key 04804ee211f5a42524d1f8340564d6a7
I20260925 07:26:10.507400 14007 raft_consensus.cc:504] T 00000000000000000000000000000000 P 054bbea9422f4472bb2b431b5088181c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "054bbea9422f4472bb2b431b5088181c" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 42669 } }
I20260925 07:26:10.507669 14007 leader_election.cc:302] T 00000000000000000000000000000000 P 054bbea9422f4472bb2b431b5088181c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 054bbea9422f4472bb2b431b5088181c; no voters:
I20260925 07:26:10.507846 14007 leader_election.cc:288] T 00000000000000000000000000000000 P 054bbea9422f4472bb2b431b5088181c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 07:26:10.507872 14012 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 054bbea9422f4472bb2b431b5088181c [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:26:10.507966 14012 raft_consensus.cc:686] T 00000000000000000000000000000000 P 054bbea9422f4472bb2b431b5088181c [term 1 LEADER]: Becoming Leader. State: Replica: 054bbea9422f4472bb2b431b5088181c, State: Running, Role: LEADER
I20260925 07:26:10.508093 14007 sys_catalog.cc:564] T 00000000000000000000000000000000 P 054bbea9422f4472bb2b431b5088181c [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:26:10.508054 14012 consensus_queue.cc:237] T 00000000000000000000000000000000 P 054bbea9422f4472bb2b431b5088181c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "054bbea9422f4472bb2b431b5088181c" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 42669 } }
I20260925 07:26:10.508657 14014 sys_catalog.cc:455] T 00000000000000000000000000000000 P 054bbea9422f4472bb2b431b5088181c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "054bbea9422f4472bb2b431b5088181c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "054bbea9422f4472bb2b431b5088181c" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 42669 } } }
I20260925 07:26:10.508744 14014 sys_catalog.cc:458] T 00000000000000000000000000000000 P 054bbea9422f4472bb2b431b5088181c [sys.catalog]: This master's current role is: LEADER
I20260925 07:26:10.508975 14013 sys_catalog.cc:455] T 00000000000000000000000000000000 P 054bbea9422f4472bb2b431b5088181c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 054bbea9422f4472bb2b431b5088181c. Latest consensus state: current_term: 1 leader_uuid: "054bbea9422f4472bb2b431b5088181c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "054bbea9422f4472bb2b431b5088181c" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 42669 } } }
I20260925 07:26:10.509073 14013 sys_catalog.cc:458] T 00000000000000000000000000000000 P 054bbea9422f4472bb2b431b5088181c [sys.catalog]: This master's current role is: LEADER
W20260925 07:26:10.510072 14027 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 054bbea9422f4472bb2b431b5088181c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 07:26:10.510114 14027 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 07:26:10.510185 14020 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 07:26:10.510368 14020 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 07:26:10.511821 14020 catalog_manager.cc:1347] Generated new cluster ID: 73671da1add2483a8a559aa0306b5541
I20260925 07:26:10.511886 14020 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 07:26:10 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321170, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:26:10.516464 14020 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 07:26:10.517150 14020 catalog_manager.cc:1504] Loading token signing keys...
I20260925 07:26:10.518245 14011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:10.507368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42010 (local address 127.5.158.126:42669)
0925 07:26:10.507527 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:10.507537 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:10.507571 (+ 34us) server_negotiation.cc:407] Connection header received
0925 07:26:10.507680 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:10.507686 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:10.507723 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:10.507873 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:10.508768 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:10.510228 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:10.511446 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:10.512069 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:10.512180 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:10.515126 (+ 2946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:26:10.515129 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:26:10.515130 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:26:10.515140 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:26:10.516758 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:26:10.517116 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:26:10.517119 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:26:10.517120 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:26:10.517260 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:26:10.517456 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:26:10.517457 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:26:10.517458 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:26:10.517680 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:26:10.517862 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0925 07:26:10.517949 (+ 87us) server_negotiation.cc:299] Negotiation successful
0925 07:26:10.518083 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":79,"threads_started":1}
I20260925 07:26:10.535772 14020 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 054bbea9422f4472bb2b431b5088181c: Generated new TSK 0
I20260925 07:26:10.535928 14020 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.158.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab.
Entry for principal kudu/127.5.158.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab.
I20260925 07:26:10.561765 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.65:0
--local_ip_for_outbound_sockets=127.5.158.65
--webserver_interface=127.5.158.65
--webserver_port=0
--tserver_master_addrs=127.5.158.126:42669
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab
--principal=kudu/127.5.158.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:41139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:26:10.635699 14033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:26:10.635903 14033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:26:10.635942 14033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:26:10.637259 14033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:26:10.637311 14033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:26:10.637352 14033 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.65
I20260925 07:26:10.638731 14033 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:41139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab
--principal=kudu/127.5.158.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.158.65
--webserver_port=0
--tserver_master_addrs=127.5.158.126:42669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14033
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.65
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:26:10.638983 14033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:26:10.639204 14033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:26:10.644047 14039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:26:10.644016 14038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:26:10.644079 14033 server_base.cc:1034] running on GCE node
W20260925 07:26:10.644019 14041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:26:10.644536 14033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:26:10.644752 14033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:26:10.645905 14033 hybrid_clock.cc:648] HybridClock initialized: now 1790321170645890 us; error 31 us; skew 500 ppm
Sep 25 07:26:10 dist-test-slave-1drq krb5kdc[13926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321170, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:26:10.648296 14033 init.cc:377] Logged in from keytab as kudu/127.5.158.65@KRBTEST.COM (short username kudu)
I20260925 07:26:10.648720 14033 webserver.cc:460] Webserver started at http://127.5.158.65:40159/ using document root <none> and password file <none>
I20260925 07:26:10.648957 14033 fs_manager.cc:359] Metadata directory not provided
I20260925 07:26:10.649029 14033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:26:10.649209 14033 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:26:10.650216 14033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-0/data/instance:
uuid: "89b81a9127cf491198403f90cda4b308"
format_stamp: "Formatted at 2026-09-25 07:26:10 on dist-test-slave-1drq"
server_key: "8ef2f67437aff4e85c9b04960df238aa"
server_key_iv: "a1a120cc1d255cf14226eec7f0610857"
server_key_version: "encryptionkey@0"
I20260925 07:26:10.650600 14033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-0/wal/instance:
uuid: "89b81a9127cf491198403f90cda4b308"
format_stamp: "Formatted at 2026-09-25 07:26:10 on dist-test-slave-1drq"
server_key: "8ef2f67437aff4e85c9b04960df238aa"
server_key_iv: "a1a120cc1d255cf14226eec7f0610857"
server_key_version: "encryptionkey@0"
I20260925 07:26:10.651948 14033 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 07:26:10.652642 14048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:10.652801 14033 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 07:26:10.652899 14033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-0/wal
uuid: "89b81a9127cf491198403f90cda4b308"
format_stamp: "Formatted at 2026-09-25 07:26:10 on dist-test-slave-1drq"
server_key: "8ef2f67437aff4e85c9b04960df238aa"
server_key_iv: "a1a120cc1d255cf14226eec7f0610857"
server_key_version: "encryptionkey@0"
I20260925 07:26:10.652984 14033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:26:10.665604 14033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:26:10.667304 14033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:26:10.667467 14033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:26:10.667689 14033 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:26:10.667996 14033 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:26:10.668056 14033 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:10.668094 14033 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:26:10.668128 14033 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:10.674098 14033 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.65:45193
I20260925 07:26:10.674134 14161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.65:45193 every 8 connection(s)
I20260925 07:26:10.674433 14033 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
I20260925 07:26:10.675644 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 14033
I20260925 07:26:10.675765 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-0/wal/instance
I20260925 07:26:10.676003 5753 external_mini_cluster.cc:1428] Setting key a4d8dc5e1d85dec276b12ebc27d81280
Sep 25 07:26:10 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321170, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.65@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.158.66@KRBTEST.COM; defaulting to no policy
I20260925 07:26:10.685288 14011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:10.675109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:33601 (local address 127.5.158.126:42669)
0925 07:26:10.675172 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:10.675174 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:10.675366 (+ 192us) server_negotiation.cc:407] Connection header received
0925 07:26:10.675861 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:10.675863 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:10.675880 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:10.675956 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:10.676982 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:10.677792 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:10.679218 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:10.679448 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:10.679514 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:10.682085 (+ 2571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:26:10.682087 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:26:10.682087 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:26:10.682096 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:26:10.683636 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:26:10.684093 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:26:10.684094 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:26:10.684095 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:26:10.684189 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:26:10.684481 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:26:10.684482 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:26:10.684482 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:26:10.684627 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:26:10.684730 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0925 07:26:10.685063 (+ 333us) server_negotiation.cc:299] Negotiation successful
0925 07:26:10.685182 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 07:26:10.685437 14164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:10.675090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42669 (local address 127.5.158.65:33601)
0925 07:26:10.675243 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:10.675291 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:10.675729 (+ 438us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:10.676009 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:10.676012 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:10.676712 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:10.676892 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:10.676896 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:10.677892 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:10.677893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:10.679127 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:10.679130 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:10.679544 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:10.679545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:10.679689 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:10.680199 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:10.680208 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:10.681976 (+ 1768us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:26:10.683748 (+ 1772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:26:10.683755 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:26:10.683757 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:26:10.683994 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:26:10.684279 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:26:10.684279 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:26:10.684280 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:26:10.684404 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:26:10.684722 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:26:10.684728 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:26:10.684871 (+ 143us) client_negotiation.cc:769] Sending connection context
0925 07:26:10.685035 (+ 164us) client_negotiation.cc:240] Negotiation successful
0925 07:26:10.685214 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":73,"threads_started":1}
I20260925 07:26:10.685930 14162 heartbeater.cc:347] Connected to a master server at 127.5.158.126:42669
I20260925 07:26:10.686020 14162 heartbeater.cc:464] Registering TS with master...
I20260925 07:26:10.686193 14162 heartbeater.cc:511] Master 127.5.158.126:42669 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.66@KRBTEST.COM" created.
I20260925 07:26:10.686728 13971 ts_manager.cc:194] Registered new tserver with Master: 89b81a9127cf491198403f90cda4b308 (127.5.158.65:45193)
I20260925 07:26:10.687337 13971 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.65@KRBTEST.COM'} at 127.5.158.65:33601
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab.
Entry for principal kudu/127.5.158.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab.
I20260925 07:26:10.699791 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.66:0
--local_ip_for_outbound_sockets=127.5.158.66
--webserver_interface=127.5.158.66
--webserver_port=0
--tserver_master_addrs=127.5.158.126:42669
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab
--principal=kudu/127.5.158.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:41139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:26:10.773479 14167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:26:10.773639 14167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:26:10.773659 14167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:26:10.774730 14167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:26:10.774766 14167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:26:10.774789 14167 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.66
I20260925 07:26:10.776041 14167 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:41139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab
--principal=kudu/127.5.158.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.158.66
--webserver_port=0
--tserver_master_addrs=127.5.158.126:42669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14167
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.66
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:26:10.776237 14167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:26:10.776415 14167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:26:10.780900 14172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:26:10.780906 14175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:26:10.780995 14173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:26:10.781062 14167 server_base.cc:1034] running on GCE node
I20260925 07:26:10.781332 14167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:26:10.781507 14167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:26:10.782629 14167 hybrid_clock.cc:648] HybridClock initialized: now 1790321170782611 us; error 31 us; skew 500 ppm
Sep 25 07:26:10 dist-test-slave-1drq krb5kdc[13926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321170, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:26:10.784768 14167 init.cc:377] Logged in from keytab as kudu/127.5.158.66@KRBTEST.COM (short username kudu)
I20260925 07:26:10.785181 14167 webserver.cc:460] Webserver started at http://127.5.158.66:40771/ using document root <none> and password file <none>
I20260925 07:26:10.785375 14167 fs_manager.cc:359] Metadata directory not provided
I20260925 07:26:10.785436 14167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:26:10.785567 14167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:26:10.786453 14167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-1/data/instance:
uuid: "6a0cd238e3974971a1e109af70acd611"
format_stamp: "Formatted at 2026-09-25 07:26:10 on dist-test-slave-1drq"
server_key: "c2d6d5eb9bdb312f7616024ffaabac35"
server_key_iv: "f93cf5cacb87d3557675facbb2fdf389"
server_key_version: "encryptionkey@0"
I20260925 07:26:10.786811 14167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-1/wal/instance:
uuid: "6a0cd238e3974971a1e109af70acd611"
format_stamp: "Formatted at 2026-09-25 07:26:10 on dist-test-slave-1drq"
server_key: "c2d6d5eb9bdb312f7616024ffaabac35"
server_key_iv: "f93cf5cacb87d3557675facbb2fdf389"
server_key_version: "encryptionkey@0"
I20260925 07:26:10.788005 14167 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 07:26:10.788595 14182 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:10.788731 14167 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:10.788812 14167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-1/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-1/wal
uuid: "6a0cd238e3974971a1e109af70acd611"
format_stamp: "Formatted at 2026-09-25 07:26:10 on dist-test-slave-1drq"
server_key: "c2d6d5eb9bdb312f7616024ffaabac35"
server_key_iv: "f93cf5cacb87d3557675facbb2fdf389"
server_key_version: "encryptionkey@0"
I20260925 07:26:10.788903 14167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:26:10.808898 14167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:26:10.810979 14167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:26:10.811115 14167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:26:10.811316 14167 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:26:10.811568 14167 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:26:10.811615 14167 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:10.811650 14167 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:26:10.811674 14167 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:10.816905 14167 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.66:34555
I20260925 07:26:10.816974 14295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.66:34555 every 8 connection(s)
I20260925 07:26:10.817267 14167 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
Sep 25 07:26:10 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321170, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.66@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:26:10.823652 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 14167
I20260925 07:26:10.823756 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-1/wal/instance
I20260925 07:26:10.823953 5753 external_mini_cluster.cc:1428] Setting key e8fcffc1b1f11b055c3c2865d081861f
I20260925 07:26:10.825160 14011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:10.817945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:43081 (local address 127.5.158.126:42669)
0925 07:26:10.818011 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:10.818013 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:10.818201 (+ 188us) server_negotiation.cc:407] Connection header received
0925 07:26:10.818588 (+ 387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:10.818589 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:10.818598 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:10.818654 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:10.819431 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:10.819861 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:10.820678 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:10.820835 (+ 157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:10.820878 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:10.822640 (+ 1762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:26:10.822641 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:26:10.822641 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:26:10.822647 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:26:10.823821 (+ 1174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:26:10.824237 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:26:10.824238 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:26:10.824238 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:26:10.824297 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:26:10.824558 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:26:10.824559 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:26:10.824559 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:26:10.824679 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:26:10.824750 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 07:26:10.824989 (+ 239us) server_negotiation.cc:299] Negotiation successful
0925 07:26:10.825061 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 07:26:10.825315 14298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:10.817897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42669 (local address 127.5.158.66:43081)
0925 07:26:10.818067 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:10.818099 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:10.818510 (+ 411us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:10.818699 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:10.818702 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:10.819184 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:10.819349 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:10.819353 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:10.819926 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:10.819927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:10.820611 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:10.820612 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:10.820896 (+ 284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:10.820896 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:10.820977 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:10.821419 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:10.821429 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:10.822565 (+ 1136us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:26:10.823894 (+ 1329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:26:10.823898 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:26:10.823900 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:26:10.824170 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:26:10.824364 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:26:10.824365 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:26:10.824365 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:26:10.824480 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:26:10.824742 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:26:10.824748 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:26:10.824860 (+ 112us) client_negotiation.cc:769] Sending connection context
0925 07:26:10.824955 (+ 95us) client_negotiation.cc:240] Negotiation successful
0925 07:26:10.825120 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":76,"threads_started":1}
I20260925 07:26:10.825757 14296 heartbeater.cc:347] Connected to a master server at 127.5.158.126:42669
I20260925 07:26:10.825824 14296 heartbeater.cc:464] Registering TS with master...
I20260925 07:26:10.825969 14296 heartbeater.cc:511] Master 127.5.158.126:42669 requested a full tablet report, sending...
I20260925 07:26:10.826326 13971 ts_manager.cc:194] Registered new tserver with Master: 6a0cd238e3974971a1e109af70acd611 (127.5.158.66:34555)
I20260925 07:26:10.826700 13971 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.66@KRBTEST.COM'} at 127.5.158.66:43081
WARNING: no policy specified for kudu/127.5.158.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab.
Entry for principal kudu/127.5.158.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab.
I20260925 07:26:10.847250 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.67:0
--local_ip_for_outbound_sockets=127.5.158.67
--webserver_interface=127.5.158.67
--webserver_port=0
--tserver_master_addrs=127.5.158.126:42669
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab
--principal=kudu/127.5.158.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:41139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:26:10.921550 14301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:26:10.921736 14301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:26:10.921770 14301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:26:10.922881 14301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:26:10.922943 14301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:26:10.922986 14301 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.67
I20260925 07:26:10.924252 14301 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:41139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab
--principal=kudu/127.5.158.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.158.67
--webserver_port=0
--tserver_master_addrs=127.5.158.126:42669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.67
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:26:10.924463 14301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:26:10.924666 14301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:26:10.929304 14307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:26:10.929307 14309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:26:10.929289 14306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:26:10.929334 14301 server_base.cc:1034] running on GCE node
I20260925 07:26:10.929723 14301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:26:10.929944 14301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:26:10.931115 14301 hybrid_clock.cc:648] HybridClock initialized: now 1790321170931099 us; error 30 us; skew 500 ppm
Sep 25 07:26:10 dist-test-slave-1drq krb5kdc[13926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321170, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:26:10.938795 14301 init.cc:377] Logged in from keytab as kudu/127.5.158.67@KRBTEST.COM (short username kudu)
I20260925 07:26:10.939108 14301 webserver.cc:460] Webserver started at http://127.5.158.67:42941/ using document root <none> and password file <none>
I20260925 07:26:10.939271 14301 fs_manager.cc:359] Metadata directory not provided
I20260925 07:26:10.939306 14301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:26:10.939381 14301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:26:10.940205 14301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-2/data/instance:
uuid: "16ec923d891c44c497efcacc3107af6d"
format_stamp: "Formatted at 2026-09-25 07:26:10 on dist-test-slave-1drq"
server_key: "7333f8c971dee4d8d2f606b106646976"
server_key_iv: "d415f4ef3ff41b61804e6c5b14e3b7f7"
server_key_version: "encryptionkey@0"
I20260925 07:26:10.940457 14301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-2/wal/instance:
uuid: "16ec923d891c44c497efcacc3107af6d"
format_stamp: "Formatted at 2026-09-25 07:26:10 on dist-test-slave-1drq"
server_key: "7333f8c971dee4d8d2f606b106646976"
server_key_iv: "d415f4ef3ff41b61804e6c5b14e3b7f7"
server_key_version: "encryptionkey@0"
I20260925 07:26:10.941592 14301 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 07:26:10.942173 14316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:10.942334 14301 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:26:10.942389 14301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-2/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-2/wal
uuid: "16ec923d891c44c497efcacc3107af6d"
format_stamp: "Formatted at 2026-09-25 07:26:10 on dist-test-slave-1drq"
server_key: "7333f8c971dee4d8d2f606b106646976"
server_key_iv: "d415f4ef3ff41b61804e6c5b14e3b7f7"
server_key_version: "encryptionkey@0"
I20260925 07:26:10.942428 14301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:26:10.966352 14301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:26:10.968003 14301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:26:10.968102 14301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:26:10.968251 14301 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:26:10.968521 14301 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:26:10.968554 14301 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:10.968572 14301 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:26:10.968616 14301 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:10.974469 14301 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.67:44051
I20260925 07:26:10.974535 14429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.67:44051 every 8 connection(s)
I20260925 07:26:10.974803 14301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
Sep 25 07:26:10 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321170, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.67@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:26:10.981588 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 14301
I20260925 07:26:10.981724 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-2/wal/instance
I20260925 07:26:10.981953 5753 external_mini_cluster.cc:1428] Setting key 5919d2e35bf4cef2f8dc2c9b2c4e435c
I20260925 07:26:10.983593 14011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:10.975476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:44387 (local address 127.5.158.126:42669)
0925 07:26:10.975524 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:10.975526 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:10.975718 (+ 192us) server_negotiation.cc:407] Connection header received
0925 07:26:10.976142 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:10.976143 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:10.976153 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:10.976204 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:10.977052 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:10.977671 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:10.978625 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:10.978821 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:10.978902 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:10.980763 (+ 1861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:26:10.980765 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:26:10.980765 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:26:10.980772 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:26:10.982320 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:26:10.982714 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:26:10.982715 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:26:10.982715 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:26:10.982772 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:26:10.983001 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:26:10.983002 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:26:10.983003 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:26:10.983111 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:26:10.983183 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 07:26:10.983429 (+ 246us) server_negotiation.cc:299] Negotiation successful
0925 07:26:10.983498 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 07:26:10.983776 14432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:10.975459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42669 (local address 127.5.158.67:44387)
0925 07:26:10.975611 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:10.975642 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:10.976048 (+ 406us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:10.976272 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:10.976275 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:10.976818 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:10.976983 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:10.976987 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:10.977773 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:10.977774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:10.978525 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:10.978527 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:10.978912 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:10.978913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:10.979034 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:10.979406 (+ 372us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:10.979413 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:10.980685 (+ 1272us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:26:10.982417 (+ 1732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:26:10.982423 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:26:10.982426 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:26:10.982626 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:26:10.982837 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:26:10.982838 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:26:10.982839 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:26:10.982935 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:26:10.983175 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:26:10.983180 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:26:10.983304 (+ 124us) client_negotiation.cc:769] Sending connection context
0925 07:26:10.983405 (+ 101us) client_negotiation.cc:240] Negotiation successful
0925 07:26:10.983544 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":77,"threads_started":1}
I20260925 07:26:10.984241 14430 heartbeater.cc:347] Connected to a master server at 127.5.158.126:42669
I20260925 07:26:10.984336 14430 heartbeater.cc:464] Registering TS with master...
I20260925 07:26:10.984520 14430 heartbeater.cc:511] Master 127.5.158.126:42669 requested a full tablet report, sending...
I20260925 07:26:10.984898 13971 ts_manager.cc:194] Registered new tserver with Master: 16ec923d891c44c497efcacc3107af6d (127.5.158.67:44051)
I20260925 07:26:10.985301 13971 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.67@KRBTEST.COM'} at 127.5.158.67:44387
I20260925 07:26:10.993459 5753 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.5.158.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.158.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.158.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/oryx_127.5.158.65.keytab.
Entry for principal oryx/127.5.158.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/oryx_127.5.158.65.keytab.
WARNING: no policy specified for oryx/127.5.158.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.158.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.158.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/oryx_127.5.158.66.keytab.
Entry for principal oryx/127.5.158.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/oryx_127.5.158.66.keytab.
WARNING: no policy specified for oryx/127.5.158.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.158.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.158.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/oryx_127.5.158.67.keytab.
Entry for principal oryx/127.5.158.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/oryx_127.5.158.67.keytab.
I20260925 07:26:11.053128 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 14033
I20260925 07:26:11.057274 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 14167
I20260925 07:26:11.061029 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 14301
I20260925 07:26:11.064806 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 13939
I20260925 07:26:11.069414 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:42669
--webserver_interface=127.5.158.126
--webserver_port=34529
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:41139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.158.126:42669
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:26:11.144732 14439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:26:11.144955 14439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:26:11.144990 14439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:26:11.146080 14439 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:26:11.146147 14439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:26:11.146171 14439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:26:11.146193 14439 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:26:11.146219 14439 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:26:11.147650 14439 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:41139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.158.126:42669
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:42669
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.158.126
--webserver_port=34529
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14439
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:26:11.147922 14439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:26:11.148130 14439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:26:11.153156 14447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:26:11.153260 14439 server_base.cc:1034] running on GCE node
W20260925 07:26:11.153272 14444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:26:11.153280 14445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:26:11.153644 14439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:26:11.153918 14439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:26:11.155093 14439 hybrid_clock.cc:648] HybridClock initialized: now 1790321171155089 us; error 31 us; skew 500 ppm
Sep 25 07:26:11 dist-test-slave-1drq krb5kdc[13926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321171, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:26:11.157548 14439 init.cc:377] Logged in from keytab as kudu/127.5.158.126@KRBTEST.COM (short username kudu)
I20260925 07:26:11.157936 14439 webserver.cc:460] Webserver started at http://127.5.158.126:34529/ using document root <none> and password file <none>
I20260925 07:26:11.158156 14439 fs_manager.cc:359] Metadata directory not provided
I20260925 07:26:11.158233 14439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:26:11.159507 14439 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 07:26:11.160120 14454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:11.160283 14439 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:26:11.160377 14439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "054bbea9422f4472bb2b431b5088181c"
format_stamp: "Formatted at 2026-09-25 07:26:10 on dist-test-slave-1drq"
server_key: "2eaa64c83bdf8e0f0efbd21e2f4efc8d"
server_key_iv: "0881ec14a58ce5131b39b9cb9defa2d0"
server_key_version: "encryptionkey@0"
I20260925 07:26:11.160701 14439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:26:11.169869 14439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:26:11.171921 14439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:26:11.172086 14439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:26:11.175894 14439 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.126:42669
I20260925 07:26:11.175925 14506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.126:42669 every 8 connection(s)
I20260925 07:26:11.176257 14439 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:26:11.177706 14507 sys_catalog.cc:263] Verifying existing consensus state
I20260925 07:26:11.178326 14507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 054bbea9422f4472bb2b431b5088181c: Bootstrap starting.
I20260925 07:26:11.183061 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 14439
I20260925 07:26:11.183431 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.65:45193
--local_ip_for_outbound_sockets=127.5.158.65
--tserver_master_addrs=127.5.158.126:42669
--webserver_port=40159
--webserver_interface=127.5.158.65
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab
--principal=kudu/127.5.158.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:41139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.5.158.65
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/oryx_127.5.158.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 07:26:11.186960 14507 log.cc:824] T 00000000000000000000000000000000 P 054bbea9422f4472bb2b431b5088181c: Log is configured to *not* fsync() on all Append() calls
I20260925 07:26:11.188376 14507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 054bbea9422f4472bb2b431b5088181c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 07:26:11.188627 14507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 054bbea9422f4472bb2b431b5088181c: Bootstrap complete.
I20260925 07:26:11.190749 14507 raft_consensus.cc:348] T 00000000000000000000000000000000 P 054bbea9422f4472bb2b431b5088181c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "054bbea9422f4472bb2b431b5088181c" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 42669 } }
I20260925 07:26:11.190961 14507 raft_consensus.cc:729] T 00000000000000000000000000000000 P 054bbea9422f4472bb2b431b5088181c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 054bbea9422f4472bb2b431b5088181c, State: Initialized, Role: FOLLOWER
I20260925 07:26:11.191104 14507 consensus_queue.cc:260] T 00000000000000000000000000000000 P 054bbea9422f4472bb2b431b5088181c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "054bbea9422f4472bb2b431b5088181c" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 42669 } }
I20260925 07:26:11.191171 14507 raft_consensus.cc:388] T 00000000000000000000000000000000 P 054bbea9422f4472bb2b431b5088181c [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 07:26:11.191196 14507 raft_consensus.cc:482] T 00000000000000000000000000000000 P 054bbea9422f4472bb2b431b5088181c [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 07:26:11.191259 14507 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 054bbea9422f4472bb2b431b5088181c [term 1 FOLLOWER]: Advancing to term 2
I20260925 07:26:11.192167 14507 raft_consensus.cc:504] T 00000000000000000000000000000000 P 054bbea9422f4472bb2b431b5088181c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "054bbea9422f4472bb2b431b5088181c" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 42669 } }
I20260925 07:26:11.192250 14507 leader_election.cc:302] T 00000000000000000000000000000000 P 054bbea9422f4472bb2b431b5088181c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 054bbea9422f4472bb2b431b5088181c; no voters:
I20260925 07:26:11.192436 14507 leader_election.cc:288] T 00000000000000000000000000000000 P 054bbea9422f4472bb2b431b5088181c [CANDIDATE]: Term 2 election: Requested vote from peers
I20260925 07:26:11.192468 14511 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 054bbea9422f4472bb2b431b5088181c [term 2 FOLLOWER]: Leader election won for term 2
I20260925 07:26:11.192596 14511 raft_consensus.cc:686] T 00000000000000000000000000000000 P 054bbea9422f4472bb2b431b5088181c [term 2 LEADER]: Becoming Leader. State: Replica: 054bbea9422f4472bb2b431b5088181c, State: Running, Role: LEADER
I20260925 07:26:11.192710 14507 sys_catalog.cc:564] T 00000000000000000000000000000000 P 054bbea9422f4472bb2b431b5088181c [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:26:11.192705 14511 consensus_queue.cc:237] T 00000000000000000000000000000000 P 054bbea9422f4472bb2b431b5088181c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "054bbea9422f4472bb2b431b5088181c" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 42669 } }
I20260925 07:26:11.192977 14511 sys_catalog.cc:455] T 00000000000000000000000000000000 P 054bbea9422f4472bb2b431b5088181c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "054bbea9422f4472bb2b431b5088181c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "054bbea9422f4472bb2b431b5088181c" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 42669 } } }
I20260925 07:26:11.193033 14511 sys_catalog.cc:458] T 00000000000000000000000000000000 P 054bbea9422f4472bb2b431b5088181c [sys.catalog]: This master's current role is: LEADER
I20260925 07:26:11.193162 14511 sys_catalog.cc:455] T 00000000000000000000000000000000 P 054bbea9422f4472bb2b431b5088181c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 054bbea9422f4472bb2b431b5088181c. Latest consensus state: current_term: 2 leader_uuid: "054bbea9422f4472bb2b431b5088181c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "054bbea9422f4472bb2b431b5088181c" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 42669 } } }
I20260925 07:26:11.193218 14511 sys_catalog.cc:458] T 00000000000000000000000000000000 P 054bbea9422f4472bb2b431b5088181c [sys.catalog]: This master's current role is: LEADER
I20260925 07:26:11.194538 14527 catalog_manager.cc:1259] Loaded cluster ID: 73671da1add2483a8a559aa0306b5541
I20260925 07:26:11.194591 14527 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 054bbea9422f4472bb2b431b5088181c: loading cluster ID for follower catalog manager: success
I20260925 07:26:11.195329 14527 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 054bbea9422f4472bb2b431b5088181c: acquiring CA information for follower catalog manager: success
I20260925 07:26:11.195590 14527 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 054bbea9422f4472bb2b431b5088181c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 07:26:11.195667 14515 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 07:26:11.195781 14515 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 07:26:11.195851 14515 catalog_manager.cc:1259] Loaded cluster ID: 73671da1add2483a8a559aa0306b5541
I20260925 07:26:11.195880 14515 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 07:26:11.196046 14515 catalog_manager.cc:1504] Loading token signing keys...
I20260925 07:26:11.196137 14515 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 054bbea9422f4472bb2b431b5088181c: Loaded TSK: 0
I20260925 07:26:11.196272 14515 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 07:26:11.281715 14509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:26:11.281972 14509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:26:11.282008 14509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:26:11.283151 14509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:26:11.283207 14509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:26:11.283250 14509 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.65
I20260925 07:26:11.284613 14509 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:41139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/oryx_127.5.158.65.keytab
--principal=oryx/127.5.158.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.65:45193
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.158.65
--webserver_port=40159
--tserver_master_addrs=127.5.158.126:42669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14509
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.65
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:26:11.284842 14509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:26:11.285043 14509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:26:11.289785 14533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:26:11.289851 14509 server_base.cc:1034] running on GCE node
W20260925 07:26:11.289868 14536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:26:11.289799 14534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:26:11.290234 14509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:26:11.290491 14509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:26:11.291648 14509 hybrid_clock.cc:648] HybridClock initialized: now 1790321171291637 us; error 29 us; skew 500 ppm
Sep 25 07:26:11 dist-test-slave-1drq krb5kdc[13926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321171, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.158.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:26:11.294521 14509 init.cc:377] Logged in from keytab as oryx/127.5.158.65@KRBTEST.COM (short username oryx)
I20260925 07:26:11.295068 14509 webserver.cc:460] Webserver started at http://127.5.158.65:40159/ using document root <none> and password file <none>
I20260925 07:26:11.295346 14509 fs_manager.cc:359] Metadata directory not provided
I20260925 07:26:11.295425 14509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:26:11.297259 14509 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 07:26:11.297941 14543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:11.298182 14509 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 07:26:11.298292 14509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-0/wal
uuid: "89b81a9127cf491198403f90cda4b308"
format_stamp: "Formatted at 2026-09-25 07:26:10 on dist-test-slave-1drq"
server_key: "8ef2f67437aff4e85c9b04960df238aa"
server_key_iv: "a1a120cc1d255cf14226eec7f0610857"
server_key_version: "encryptionkey@0"
I20260925 07:26:11.298691 14509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:26:11.322768 14509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:26:11.325014 14509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:26:11.325218 14509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:26:11.325487 14509 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:26:11.325845 14509 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:26:11.325891 14509 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:11.325918 14509 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:26:11.325940 14509 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:11.333814 14509 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.65:45193
I20260925 07:26:11.333842 14656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.65:45193 every 8 connection(s)
I20260925 07:26:11.334255 14509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
I20260925 07:26:11.338656 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 14509
I20260925 07:26:11.338974 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.66:34555
--local_ip_for_outbound_sockets=127.5.158.66
--tserver_master_addrs=127.5.158.126:42669
--webserver_port=40771
--webserver_interface=127.5.158.66
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab
--principal=kudu/127.5.158.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:41139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.5.158.66
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/oryx_127.5.158.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 07:26:11 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.65@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 07:26:11 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.65@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 07:26:11.343959 14659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:11.334971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42669 (local address 127.5.158.65:49993)
0925 07:26:11.335167 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:11.335215 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:11.335874 (+ 659us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:11.336282 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:11.336294 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:11.337137 (+ 843us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:11.337461 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:11.337468 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:11.338544 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:11.338545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:11.339500 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:11.339503 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:11.340071 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:11.340072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:11.340208 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:11.340939 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:11.340950 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:11.343754 (+ 2804us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.158.126:42669: Server oryx/127.5.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":87,"threads_started":1}
W20260925 07:26:11.344137 14660 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:26:11.335097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:49993 (local address 127.5.158.126:42669)
0925 07:26:11.335274 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:11.335281 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:11.335306 (+ 25us) server_negotiation.cc:407] Connection header received
0925 07:26:11.336054 (+ 748us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:11.336059 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:11.336096 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:11.336194 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:11.337568 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:11.338423 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:11.339592 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:11.339978 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:11.340095 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:11.343919 (+ 3824us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.158.65:49993: BlockingRecv error: recv got EOF from 127.5.158.65:49993 (error 108)
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":97,"threads_started":1}
W20260925 07:26:11.344456 14657 heartbeater.cc:644] Failed to heartbeat to 127.5.158.126:42669 (0 consecutive failures): Not authorized: Failed to ping master at 127.5.158.126:42669: Client connection negotiation failed: client connection to 127.5.158.126:42669: Server oryx/127.5.158.126@KRBTEST.COM not found in Kerberos database
Sep 25 07:26:11 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.65@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 07:26:11 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.65@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 07:26:11.350450 14659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:11.344688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42669 (local address 127.5.158.65:38217)
0925 07:26:11.344829 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:11.344840 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:11.344894 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:11.345052 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:11.345053 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:11.345244 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:11.345316 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:11.345319 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:11.346334 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:11.346335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:11.347198 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:11.347201 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:11.347715 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:11.347716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:11.347787 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:11.348283 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:11.348295 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:11.350383 (+ 2088us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.158.126:42669: Server oryx/127.5.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":73,"threads_started":1}
W20260925 07:26:11.350611 14660 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:26:11.344755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:38217 (local address 127.5.158.126:42669)
0925 07:26:11.344796 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:11.344798 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:11.344899 (+ 101us) server_negotiation.cc:407] Connection header received
0925 07:26:11.344967 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:11.344968 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:11.344982 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:11.345049 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:11.345403 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:11.346226 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:11.347297 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:11.347627 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:11.347706 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:11.350536 (+ 2830us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.158.65:38217: BlockingRecv error: recv got EOF from 127.5.158.65:38217 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Sep 25 07:26:11 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.65@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 07:26:11 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.65@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 07:26:11.356221 14659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:11.351032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42669 (local address 127.5.158.65:42999)
0925 07:26:11.351072 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:11.351081 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:11.351112 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:11.351244 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:11.351244 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:11.351376 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:11.351404 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:11.351405 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:11.352256 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:11.352257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:11.353076 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:11.353077 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:11.353818 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:11.353819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:11.353879 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:11.354342 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:11.354347 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:11.356155 (+ 1808us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.158.126:42669: Server oryx/127.5.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":28}
W20260925 07:26:11.356452 14660 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:26:11.351092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:42999 (local address 127.5.158.126:42669)
0925 07:26:11.351122 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:11.351124 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:11.351132 (+ 8us) server_negotiation.cc:407] Connection header received
0925 07:26:11.351170 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:11.351171 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:11.351180 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:11.351229 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:11.351476 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:11.352177 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:11.353167 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:11.353459 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:11.353527 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:11.356378 (+ 2851us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.158.65:42999: BlockingRecv error: recv got EOF from 127.5.158.65:42999 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260925 07:26:11.356715 14657 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 07:26:11.422739 14661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:26:11.422959 14661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:26:11.422997 14661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:26:11.424466 14661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:26:11.424557 14661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:26:11.424603 14661 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.66
I20260925 07:26:11.426064 14661 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:41139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/oryx_127.5.158.66.keytab
--principal=oryx/127.5.158.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.66:34555
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.158.66
--webserver_port=40771
--tserver_master_addrs=127.5.158.126:42669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14661
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.66
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:26:11.426321 14661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:26:11.426520 14661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:26:11.431419 14667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:26:11.431442 14668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:26:11.431488 14661 server_base.cc:1034] running on GCE node
W20260925 07:26:11.431463 14670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:26:11.431879 14661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:26:11.432116 14661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:26:11.433267 14661 hybrid_clock.cc:648] HybridClock initialized: now 1790321171433252 us; error 49 us; skew 500 ppm
Sep 25 07:26:11 dist-test-slave-1drq krb5kdc[13926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321171, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.158.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:26:11.435611 14661 init.cc:377] Logged in from keytab as oryx/127.5.158.66@KRBTEST.COM (short username oryx)
I20260925 07:26:11.436043 14661 webserver.cc:460] Webserver started at http://127.5.158.66:40771/ using document root <none> and password file <none>
I20260925 07:26:11.436287 14661 fs_manager.cc:359] Metadata directory not provided
I20260925 07:26:11.436339 14661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:26:11.437719 14661 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 07:26:11.438313 14677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:11.438520 14661 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 07:26:11.438617 14661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-1/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-1/wal
uuid: "6a0cd238e3974971a1e109af70acd611"
format_stamp: "Formatted at 2026-09-25 07:26:10 on dist-test-slave-1drq"
server_key: "c2d6d5eb9bdb312f7616024ffaabac35"
server_key_iv: "f93cf5cacb87d3557675facbb2fdf389"
server_key_version: "encryptionkey@0"
I20260925 07:26:11.438910 14661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:26:11.455093 14661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:26:11.456722 14661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:26:11.456867 14661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:26:11.457067 14661 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:26:11.457412 14661 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:26:11.457446 14661 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:11.457465 14661 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:26:11.457516 14661 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:11.462890 14661 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.66:34555
I20260925 07:26:11.462936 14790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.66:34555 every 8 connection(s)
I20260925 07:26:11.463215 14661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
I20260925 07:26:11.464339 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 14661
I20260925 07:26:11.464601 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.67:44051
--local_ip_for_outbound_sockets=127.5.158.67
--tserver_master_addrs=127.5.158.126:42669
--webserver_port=42941
--webserver_interface=127.5.158.67
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab
--principal=kudu/127.5.158.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:41139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.5.158.67
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/oryx_127.5.158.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 07:26:11 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.66@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 07:26:11 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.66@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 07:26:11.470736 14793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:11.463741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42669 (local address 127.5.158.66:43637)
0925 07:26:11.463937 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:11.464002 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:11.464568 (+ 566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:11.464813 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:11.464816 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:11.465419 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:11.465596 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:11.465600 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:11.466597 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:11.466598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:11.467379 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:11.467381 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:11.467899 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:11.467899 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:11.467993 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:11.468547 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:11.468555 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:11.470609 (+ 2054us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.158.126:42669: Server oryx/127.5.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":126,"threads_started":1}
W20260925 07:26:11.470806 14660 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:26:11.463770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:43637 (local address 127.5.158.126:42669)
0925 07:26:11.463809 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:11.463811 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:11.464100 (+ 289us) server_negotiation.cc:407] Connection header received
0925 07:26:11.464682 (+ 582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:11.464684 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:11.464701 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:11.464751 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:11.465699 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:11.466493 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:11.467470 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:11.467814 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:11.467891 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:11.470726 (+ 2835us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.158.66:43637: BlockingRecv error: recv got EOF from 127.5.158.66:43637 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260925 07:26:11.470973 14791 heartbeater.cc:644] Failed to heartbeat to 127.5.158.126:42669 (0 consecutive failures): Not authorized: Failed to ping master at 127.5.158.126:42669: Client connection negotiation failed: client connection to 127.5.158.126:42669: Server oryx/127.5.158.126@KRBTEST.COM not found in Kerberos database
Sep 25 07:26:11 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.66@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 07:26:11 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.66@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 07:26:11.475855 14795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:11.471161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42669 (local address 127.5.158.66:60243)
0925 07:26:11.471320 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:11.471330 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:11.471401 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:11.471523 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:11.471524 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:11.471698 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:11.471756 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:11.471757 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:11.472326 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:11.472327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:11.473277 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:11.473280 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:11.473621 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:11.473623 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:11.473693 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:11.474061 (+ 368us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:11.474066 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:11.475778 (+ 1712us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.158.126:42669: Server oryx/127.5.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":77,"threads_started":1}
W20260925 07:26:11.475919 14660 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:26:11.471243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:60243 (local address 127.5.158.126:42669)
0925 07:26:11.471284 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:11.471286 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:11.471399 (+ 113us) server_negotiation.cc:407] Connection header received
0925 07:26:11.471461 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:11.471462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:11.471469 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:11.471514 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:11.471828 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:11.472252 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:11.473372 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:11.473533 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:11.473582 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:11.475859 (+ 2277us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.158.66:60243: BlockingRecv error: recv got EOF from 127.5.158.66:60243 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Sep 25 07:26:11 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.66@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 07:26:11 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.66@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 07:26:11.480095 14795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:11.476052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42669 (local address 127.5.158.66:33735)
0925 07:26:11.476094 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:11.476103 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:11.476131 (+ 28us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:11.476263 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:11.476264 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:11.476403 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:11.476432 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:11.476433 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:11.476996 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:11.476997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:11.477832 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:11.477834 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:11.478108 (+ 274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:11.478108 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:11.478163 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:11.478470 (+ 307us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:11.478475 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:11.480010 (+ 1535us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.158.126:42669: Server oryx/127.5.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":34}
W20260925 07:26:11.480141 14660 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:26:11.476102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:33735 (local address 127.5.158.126:42669)
0925 07:26:11.476130 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:11.476131 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:11.476139 (+ 8us) server_negotiation.cc:407] Connection header received
0925 07:26:11.476181 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:11.476182 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:11.476192 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:11.476241 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:11.476494 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:11.476927 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:11.477897 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:11.478046 (+ 149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:11.478092 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:11.480084 (+ 1992us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.158.66:33735: BlockingRecv error: recv got EOF from 127.5.158.66:33735 (error 108)
Metrics: {"server-negotiator.queue_time_us":14}
W20260925 07:26:11.480362 14791 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 07:26:11.545847 14794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:26:11.546021 14794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:26:11.546042 14794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:26:11.547170 14794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:26:11.547209 14794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:26:11.547263 14794 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.67
I20260925 07:26:11.548619 14794 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:41139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/krb5kdc/oryx_127.5.158.67.keytab
--principal=oryx/127.5.158.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.67:44051
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.158.67
--webserver_port=42941
--tserver_master_addrs=127.5.158.126:42669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14794
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.67
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:26:11.548874 14794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:26:11.549098 14794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 07:26:11.554035 14794 server_base.cc:1034] running on GCE node
W20260925 07:26:11.553999 14801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:26:11.554087 14800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:26:11.554128 14803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:26:11.554435 14794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:26:11.554683 14794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:26:11.555845 14794 hybrid_clock.cc:648] HybridClock initialized: now 1790321171555830 us; error 31 us; skew 500 ppm
Sep 25 07:26:11 dist-test-slave-1drq krb5kdc[13926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321171, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.158.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:26:11.558326 14794 init.cc:377] Logged in from keytab as oryx/127.5.158.67@KRBTEST.COM (short username oryx)
I20260925 07:26:11.558720 14794 webserver.cc:460] Webserver started at http://127.5.158.67:42941/ using document root <none> and password file <none>
I20260925 07:26:11.558882 14794 fs_manager.cc:359] Metadata directory not provided
I20260925 07:26:11.558933 14794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:26:11.560122 14794 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 07:26:11.560638 14810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:11.560806 14794 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:11.560886 14794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-2/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-2/wal
uuid: "16ec923d891c44c497efcacc3107af6d"
format_stamp: "Formatted at 2026-09-25 07:26:10 on dist-test-slave-1drq"
server_key: "7333f8c971dee4d8d2f606b106646976"
server_key_iv: "d415f4ef3ff41b61804e6c5b14e3b7f7"
server_key_version: "encryptionkey@0"
I20260925 07:26:11.561213 14794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:26:11.577486 14794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:26:11.579203 14794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:26:11.579303 14794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:26:11.579557 14794 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:26:11.579828 14794 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:26:11.579879 14794 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:11.579916 14794 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:26:11.579941 14794 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:11.585449 14794 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.67:44051
I20260925 07:26:11.585487 14923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.67:44051 every 8 connection(s)
I20260925 07:26:11.585821 14794 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
I20260925 07:26:11.589560 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 14794
Sep 25 07:26:11 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.67@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 07:26:11 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.67@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
W20260925 07:26:11.593066 14660 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:26:11.586492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:45361 (local address 127.5.158.126:42669)
0925 07:26:11.586538 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:11.586540 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:11.586752 (+ 212us) server_negotiation.cc:407] Connection header received
0925 07:26:11.587236 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:11.587238 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:11.587250 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:11.587297 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:11.588164 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:11.588643 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:11.589669 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:11.589903 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:11.589969 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:11.593011 (+ 3042us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.158.67:45361: BlockingRecv error: recv got EOF from 127.5.158.67:45361 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 07:26:11.593149 14926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:11.586469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42669 (local address 127.5.158.67:45361)
0925 07:26:11.586626 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:11.586662 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:11.587130 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:11.587363 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:11.587373 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:11.587905 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:11.588066 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:11.588069 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:11.588743 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:11.588745 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:11.589583 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:11.589586 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:11.589993 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:11.589994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:11.590126 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:11.590579 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:11.590593 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:11.592948 (+ 2355us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.158.126:42669: Server oryx/127.5.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":82,"threads_started":1}
W20260925 07:26:11.593340 14924 heartbeater.cc:644] Failed to heartbeat to 127.5.158.126:42669 (0 consecutive failures): Not authorized: Failed to ping master at 127.5.158.126:42669: Client connection negotiation failed: client connection to 127.5.158.126:42669: Server oryx/127.5.158.126@KRBTEST.COM not found in Kerberos database
Sep 25 07:26:11 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.67@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 07:26:11 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.67@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 07:26:11.598292 14926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:11.593556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42669 (local address 127.5.158.67:37371)
0925 07:26:11.593603 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:11.593613 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:11.593651 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:11.593871 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:11.593872 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:11.594007 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:11.594055 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:11.594057 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:11.594919 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:11.594919 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:11.595431 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:11.595433 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:11.595917 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:11.595917 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:11.596031 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:11.596388 (+ 357us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:11.596395 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:11.598235 (+ 1840us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.158.126:42669: Server oryx/127.5.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":20}
W20260925 07:26:11.598398 14660 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:26:11.593580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:37371 (local address 127.5.158.126:42669)
0925 07:26:11.593627 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:11.593629 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:11.593641 (+ 12us) server_negotiation.cc:407] Connection header received
0925 07:26:11.593753 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:11.593754 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:11.593765 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:11.593846 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:11.594133 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:11.594840 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:11.595535 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:11.595841 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:11.595920 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:11.598328 (+ 2408us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.158.67:37371: BlockingRecv error: recv got EOF from 127.5.158.67:37371 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
I20260925 07:26:11.599103 14928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:11.590040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42024 (local address 127.5.158.126:42669)
0925 07:26:11.590146 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:11.590147 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:11.590245 (+ 98us) server_negotiation.cc:407] Connection header received
0925 07:26:11.590318 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:11.590319 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:11.590330 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:11.590402 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:11.591331 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:11.591994 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:11.592984 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:11.593287 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:11.593367 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:11.595078 (+ 1711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:26:11.595082 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:26:11.595083 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:26:11.595093 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:26:11.597043 (+ 1950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:26:11.597504 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:26:11.597507 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:26:11.597507 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:26:11.597668 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:26:11.597894 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:26:11.597895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:26:11.597896 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:26:11.598155 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:26:11.598304 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0925 07:26:11.598880 (+ 576us) server_negotiation.cc:299] Negotiation successful
0925 07:26:11.599013 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":49,"threads_started":1}
Sep 25 07:26:11 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.67@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 07:26:11 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.67@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 07:26:11.602627 14926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:11.598762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42669 (local address 127.5.158.67:53717)
0925 07:26:11.598846 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:11.598855 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:11.598894 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:11.599017 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:11.599017 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:11.599105 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:11.599126 (+ 21us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:11.599127 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:11.599903 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:11.599904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:11.600407 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:11.600408 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:11.600728 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:11.600728 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:11.600760 (+ 32us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:11.601141 (+ 381us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:11.601145 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:11.602592 (+ 1447us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.158.126:42669: Server oryx/127.5.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":70}
W20260925 07:26:11.602730 14660 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:26:11.598811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:53717 (local address 127.5.158.126:42669)
0925 07:26:11.598846 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:11.598848 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:11.598902 (+ 54us) server_negotiation.cc:407] Connection header received
0925 07:26:11.598924 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:11.598925 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:11.598933 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:11.599006 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:11.599245 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:11.599827 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:11.600503 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:11.600664 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:11.600718 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:11.602662 (+ 1944us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.158.67:53717: BlockingRecv error: recv got EOF from 127.5.158.67:53717 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260925 07:26:11.602770 14924 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 25 07:26:12 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.65@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 07:26:12 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.65@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 07:26:12.361385 14929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:12.357117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42669 (local address 127.5.158.65:40401)
0925 07:26:12.357262 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:12.357273 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:12.357318 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:12.357470 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:12.357471 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:12.357614 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:12.357660 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:12.357662 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:12.358250 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:12.358251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:12.358723 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:12.358724 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:12.359003 (+ 279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:12.359004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:12.359041 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:12.359481 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:12.359485 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:12.361329 (+ 1844us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.158.126:42669: Server oryx/127.5.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":85,"threads_started":1}
W20260925 07:26:12.361492 14930 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:26:12.357219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:40401 (local address 127.5.158.126:42669)
0925 07:26:12.357336 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:12.357338 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:12.357353 (+ 15us) server_negotiation.cc:407] Connection header received
0925 07:26:12.357375 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:12.357376 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:12.357385 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:12.357453 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:12.357719 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:12.358188 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:12.358780 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:12.358945 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:12.358989 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:12.361417 (+ 2428us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.158.65:40401: BlockingRecv error: recv got EOF from 127.5.158.65:40401 (error 108)
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":76,"threads_started":1}
Sep 25 07:26:12 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.66@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 07:26:12 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.66@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 07:26:12.485462 14931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:12.480964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42669 (local address 127.5.158.66:50553)
0925 07:26:12.481153 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:12.481164 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:12.481241 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:12.481381 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:12.481382 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:12.481551 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:12.481592 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:12.481594 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:12.482210 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:12.482211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:12.482701 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:12.482703 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:12.483036 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:12.483037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:12.483090 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:12.483545 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:12.483550 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:12.485349 (+ 1799us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.158.126:42669: Server oryx/127.5.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":80,"threads_started":1}
W20260925 07:26:12.485497 14930 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:26:12.481053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:50553 (local address 127.5.158.126:42669)
0925 07:26:12.481120 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:12.481122 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:12.481224 (+ 102us) server_negotiation.cc:407] Connection header received
0925 07:26:12.481298 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:12.481299 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:12.481313 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:12.481375 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:12.481670 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:12.482130 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:12.482786 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:12.482967 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:12.483038 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:12.485454 (+ 2416us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.158.66:50553: BlockingRecv error: recv got EOF from 127.5.158.66:50553 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Sep 25 07:26:12 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.67@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 07:26:12 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.67@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 07:26:12.607620 14932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:12.603175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42669 (local address 127.5.158.67:41839)
0925 07:26:12.603320 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:12.603330 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:12.603414 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:12.603567 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:12.603568 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:12.603724 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:12.603762 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:12.603763 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:12.604357 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:12.604358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:12.604823 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:12.604824 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:12.605172 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:12.605173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:12.605227 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:12.605694 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:12.605698 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:12.607558 (+ 1860us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.158.126:42669: Server oryx/127.5.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":67,"threads_started":1}
W20260925 07:26:12.607733 14930 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:26:12.603243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:41839 (local address 127.5.158.126:42669)
0925 07:26:12.603304 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:12.603306 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:12.603388 (+ 82us) server_negotiation.cc:407] Connection header received
0925 07:26:12.603471 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:12.603472 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:12.603501 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:12.603560 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:12.603833 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:12.604280 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:12.604905 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:12.605068 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:12.605171 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:12.607640 (+ 2469us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.158.67:41839: BlockingRecv error: recv got EOF from 127.5.158.67:41839 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Sep 25 07:26:13 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.65@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 07:26:13 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.65@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 07:26:13.366392 14933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:13.361828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42669 (local address 127.5.158.65:43463)
0925 07:26:13.361950 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:13.361961 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:13.362026 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:13.362174 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:13.362174 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:13.362318 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:13.362387 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:13.362389 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:13.362999 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:13.363000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:13.363573 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:13.363575 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:13.363918 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:13.363920 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:13.363977 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:13.364446 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:13.364450 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:13.366310 (+ 1860us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.158.126:42669: Server oryx/127.5.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":76,"threads_started":1}
W20260925 07:26:13.366474 14934 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:26:13.361883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:43463 (local address 127.5.158.126:42669)
0925 07:26:13.362001 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:13.362003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:13.362010 (+ 7us) server_negotiation.cc:407] Connection header received
0925 07:26:13.362075 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:13.362076 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:13.362086 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:13.362139 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:13.362452 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:13.362930 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:13.363642 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:13.363823 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:13.363873 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:13.366406 (+ 2533us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.158.65:43463: BlockingRecv error: recv got EOF from 127.5.158.65:43463 (error 108)
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":71,"threads_started":1}
Sep 25 07:26:13 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.66@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 07:26:13 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.66@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 07:26:13.490386 14935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:13.485975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42669 (local address 127.5.158.66:46689)
0925 07:26:13.486099 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:13.486109 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:13.486168 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:13.486330 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:13.486331 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:13.486470 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:13.486517 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:13.486519 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:13.487195 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:13.487196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:13.487723 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:13.487725 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:13.488073 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:13.488074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:13.488116 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:13.488523 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:13.488527 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:13.490331 (+ 1804us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.158.126:42669: Server oryx/127.5.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":83,"threads_started":1}
W20260925 07:26:13.490489 14934 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:26:13.486049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:46689 (local address 127.5.158.126:42669)
0925 07:26:13.486087 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:13.486089 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:13.486146 (+ 57us) server_negotiation.cc:407] Connection header received
0925 07:26:13.486233 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:13.486234 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:13.486242 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:13.486290 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:13.486582 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:13.487083 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:13.487810 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:13.487984 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:13.488073 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:13.490421 (+ 2348us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.158.66:46689: BlockingRecv error: recv got EOF from 127.5.158.66:46689 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
Sep 25 07:26:13 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.67@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 07:26:13 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.67@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 07:26:13.612366 14936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:13.608003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42669 (local address 127.5.158.67:50539)
0925 07:26:13.608133 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:13.608144 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:13.608195 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:13.608329 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:13.608330 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:13.608529 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:13.608588 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:13.608589 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:13.609294 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:13.609295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:13.609775 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:13.609776 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:13.610115 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:13.610116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:13.610175 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:13.610659 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:13.610663 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:13.612277 (+ 1614us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.158.126:42669: Server oryx/127.5.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":67,"threads_started":1}
W20260925 07:26:13.612391 14934 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:26:13.608110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:50539 (local address 127.5.158.126:42669)
0925 07:26:13.608170 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:13.608172 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:13.608179 (+ 7us) server_negotiation.cc:407] Connection header received
0925 07:26:13.608244 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:13.608245 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:13.608256 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:13.608322 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:13.608664 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:13.609206 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:13.609866 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:13.610037 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:13.610114 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:13.612339 (+ 2225us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.158.67:50539: BlockingRecv error: recv got EOF from 127.5.158.67:50539 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Sep 25 07:26:14 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.65@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 07:26:14 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.65@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 07:26:14.371078 14937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:14.366899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42669 (local address 127.5.158.65:52435)
0925 07:26:14.367054 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:14.367068 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:14.367130 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:14.367359 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:14.367360 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:14.367536 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:14.367587 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:14.367588 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:14.368190 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:14.368191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:14.368655 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:14.368657 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:14.368938 (+ 281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:14.368939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:14.368975 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:14.369419 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:14.369423 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:14.371027 (+ 1604us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.158.126:42669: Server oryx/127.5.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":64,"threads_started":1}
W20260925 07:26:14.371218 14938 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:26:14.366983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:52435 (local address 127.5.158.126:42669)
0925 07:26:14.367114 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:14.367116 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:14.367126 (+ 10us) server_negotiation.cc:407] Connection header received
0925 07:26:14.367234 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:14.367236 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:14.367249 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:14.367319 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:14.367660 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:14.368131 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:14.368717 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:14.368884 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:14.368927 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:14.371118 (+ 2191us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.158.65:52435: BlockingRecv error: recv got EOF from 127.5.158.65:52435 (error 108)
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":55,"threads_started":1}
Sep 25 07:26:14 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.66@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 07:26:14 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.66@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 07:26:14.495782 14939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:14.490950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42669 (local address 127.5.158.66:35665)
0925 07:26:14.491152 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:14.491170 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:14.491248 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:14.491395 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:14.491396 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:14.491560 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:14.491607 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:14.491608 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:14.492246 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:14.492247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:14.492804 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:14.492806 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:14.493285 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:14.493286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:14.493342 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:14.493750 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:14.493755 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:14.495668 (+ 1913us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.158.126:42669: Server oryx/127.5.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":86,"threads_started":1}
W20260925 07:26:14.495853 14938 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:26:14.491030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:35665 (local address 127.5.158.126:42669)
0925 07:26:14.491114 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:14.491117 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:14.491229 (+ 112us) server_negotiation.cc:407] Connection header received
0925 07:26:14.491312 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:14.491313 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:14.491325 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:14.491393 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:14.491689 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:14.492164 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:14.492894 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:14.493216 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:14.493286 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:14.495759 (+ 2473us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.158.66:35665: BlockingRecv error: recv got EOF from 127.5.158.66:35665 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Sep 25 07:26:14 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.67@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 07:26:14 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.67@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 07:26:14.616933 14940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:14.612749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42669 (local address 127.5.158.67:35503)
0925 07:26:14.612903 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:14.612913 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:14.612989 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:14.613169 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:14.613169 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:14.613313 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:14.613375 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:14.613378 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:14.614129 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:14.614130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:14.614570 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:14.614572 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:14.614848 (+ 276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:14.614848 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:14.614883 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:14.615305 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:14.615310 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:14.616847 (+ 1537us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.158.126:42669: Server oryx/127.5.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":78,"threads_started":1}
W20260925 07:26:14.616978 14938 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:26:14.612822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:35503 (local address 127.5.158.126:42669)
0925 07:26:14.612885 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:14.612887 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:14.612976 (+ 89us) server_negotiation.cc:407] Connection header received
0925 07:26:14.613038 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:14.613039 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:14.613052 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:14.613156 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:14.613448 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:14.614068 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:14.614638 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:14.614791 (+ 153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:14.614837 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:14.616915 (+ 2078us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.158.67:35503: BlockingRecv error: recv got EOF from 127.5.158.67:35503 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Sep 25 07:26:15 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.65@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 07:26:15 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.65@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 07:26:15.375808 14941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:15.371512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42669 (local address 127.5.158.65:33155)
0925 07:26:15.371641 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:15.371652 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:15.371704 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:15.371835 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:15.371836 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:15.371977 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:15.372024 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:15.372025 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:15.372599 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:15.372600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:15.373073 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:15.373074 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:15.373466 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:15.373467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:15.373521 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:15.373945 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:15.373950 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:15.375733 (+ 1783us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.158.126:42669: Server oryx/127.5.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":77,"threads_started":1}
W20260925 07:26:15.375888 14942 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:26:15.371563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:33155 (local address 127.5.158.126:42669)
0925 07:26:15.371693 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:15.371695 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:15.371702 (+ 7us) server_negotiation.cc:407] Connection header received
0925 07:26:15.371753 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:15.371754 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:15.371764 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:15.371810 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:15.372079 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:15.372530 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:15.373162 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:15.373376 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:15.373432 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:15.375826 (+ 2394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.158.65:33155: BlockingRecv error: recv got EOF from 127.5.158.65:33155 (error 108)
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":78,"threads_started":1}
Sep 25 07:26:15 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.66@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 07:26:15 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.66@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 07:26:15.500455 14943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:15.496341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42669 (local address 127.5.158.66:52979)
0925 07:26:15.496471 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:15.496481 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:15.496534 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:15.496642 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:15.496642 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:15.496806 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:15.496855 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:15.496856 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:15.497529 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:15.497530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:15.498033 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:15.498034 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:15.498304 (+ 270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:15.498304 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:15.498339 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:15.498697 (+ 358us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:15.498701 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:15.500406 (+ 1705us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.158.126:42669: Server oryx/127.5.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":86,"threads_started":1}
W20260925 07:26:15.500543 14942 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:26:15.496380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:52979 (local address 127.5.158.126:42669)
0925 07:26:15.496438 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:15.496440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:15.496533 (+ 93us) server_negotiation.cc:407] Connection header received
0925 07:26:15.496572 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:15.496573 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:15.496582 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:15.496626 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:15.496912 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:15.497466 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:15.498097 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:15.498248 (+ 151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:15.498290 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:15.500483 (+ 2193us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.158.66:52979: BlockingRecv error: recv got EOF from 127.5.158.66:52979 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Sep 25 07:26:15 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.67@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 07:26:15 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.67@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 07:26:15.621218 14944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:15.617288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42669 (local address 127.5.158.67:36933)
0925 07:26:15.617414 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:15.617424 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:15.617477 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:15.617590 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:15.617591 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:15.617732 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:15.617781 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:15.617782 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:15.618396 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:15.618397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:15.618837 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:15.618838 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:15.619121 (+ 283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:15.619122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:15.619158 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:15.619541 (+ 383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:15.619544 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:15.621166 (+ 1622us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.158.126:42669: Server oryx/127.5.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":70,"threads_started":1}
W20260925 07:26:15.621341 14942 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:26:15.617355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:36933 (local address 127.5.158.126:42669)
0925 07:26:15.617399 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:15.617401 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:15.617466 (+ 65us) server_negotiation.cc:407] Connection header received
0925 07:26:15.617517 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:15.617517 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:15.617527 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:15.617573 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:15.617843 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:15.618326 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:15.618898 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:15.619050 (+ 152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:15.619109 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:15.621259 (+ 2150us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.158.67:36933: BlockingRecv error: recv got EOF from 127.5.158.67:36933 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Sep 25 07:26:16 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.65@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 07:26:16 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.65@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 07:26:16.385751 14945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:16.376272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42669 (local address 127.5.158.65:57085)
0925 07:26:16.376399 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:16.376412 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:16.376473 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:16.376592 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:16.376593 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:16.376767 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:16.376815 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:16.376816 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:16.377759 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:16.377759 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:16.378279 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:16.378281 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:16.378627 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:16.378628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:16.378673 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:16.383783 (+ 5110us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:16.383788 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:16.385685 (+ 1897us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.158.126:42669: Server oryx/127.5.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":71,"threads_started":1}
W20260925 07:26:16.385955 14946 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:26:16.376345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:57085 (local address 127.5.158.126:42669)
0925 07:26:16.376469 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:16.376471 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:16.376479 (+ 8us) server_negotiation.cc:407] Connection header received
0925 07:26:16.376516 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:16.376517 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:16.376526 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:16.376580 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:16.376894 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:16.377686 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:16.378343 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:16.378531 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:16.378594 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:16.385841 (+ 7247us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.158.65:57085: BlockingRecv error: recv got EOF from 127.5.158.65:57085 (error 108)
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":61,"threads_started":1}
Sep 25 07:26:16 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.66@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 07:26:16 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.66@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 07:26:16.505577 14947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:16.500988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42669 (local address 127.5.158.66:49189)
0925 07:26:16.501136 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:16.501148 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:16.501225 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:16.501366 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:16.501367 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:16.501554 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:16.501616 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:16.501618 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:16.502176 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:16.502176 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:16.502709 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:16.502711 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:16.502997 (+ 286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:16.502998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:16.503041 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:16.503572 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:16.503579 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:16.505522 (+ 1943us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.158.126:42669: Server oryx/127.5.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":92,"threads_started":1}
W20260925 07:26:16.505641 14946 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:26:16.501112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:49189 (local address 127.5.158.126:42669)
0925 07:26:16.501158 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:16.501159 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:16.501200 (+ 41us) server_negotiation.cc:407] Connection header received
0925 07:26:16.501286 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:16.501287 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:16.501296 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:16.501345 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:16.501676 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:16.502120 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:16.502772 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:16.502939 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:16.502983 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:16.505597 (+ 2614us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.158.66:49189: BlockingRecv error: recv got EOF from 127.5.158.66:49189 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Sep 25 07:26:16 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.67@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 07:26:16 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.67@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 07:26:16.626192 14948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:16.621687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42669 (local address 127.5.158.67:49527)
0925 07:26:16.621812 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:16.621824 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:16.621883 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:16.622000 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:16.622000 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:16.622146 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:16.622192 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:16.622194 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:16.622914 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:16.622916 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:16.623506 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:16.623509 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:16.623801 (+ 292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:16.623802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:16.623840 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:16.624265 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:16.624272 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:16.626142 (+ 1870us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.158.126:42669: Server oryx/127.5.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":76,"threads_started":1}
W20260925 07:26:16.626323 14946 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:26:16.621771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:49527 (local address 127.5.158.126:42669)
0925 07:26:16.621824 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:16.621826 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:16.621874 (+ 48us) server_negotiation.cc:407] Connection header received
0925 07:26:16.621924 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:16.621924 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:16.621935 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:16.621983 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:16.622256 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:16.622824 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:16.623574 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:16.623740 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:16.623788 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:16.626249 (+ 2461us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.158.67:49527: BlockingRecv error: recv got EOF from 127.5.158.67:49527 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Sep 25 07:26:17 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.65@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 07:26:17 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.65@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 07:26:17.390399 14949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:17.386165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42669 (local address 127.5.158.65:54627)
0925 07:26:17.386302 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:17.386312 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:17.386356 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:17.386500 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:17.386501 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:17.386652 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:17.386707 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:17.386708 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:17.387339 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:17.387339 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:17.387849 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:17.387850 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:17.388190 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:17.388190 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:17.388226 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:17.388625 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:17.388629 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:17.390311 (+ 1682us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.158.126:42669: Server oryx/127.5.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":79,"threads_started":1}
W20260925 07:26:17.390448 14950 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:26:17.386268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:54627 (local address 127.5.158.126:42669)
0925 07:26:17.386378 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:17.386380 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:17.386395 (+ 15us) server_negotiation.cc:407] Connection header received
0925 07:26:17.386414 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:17.386415 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:17.386424 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:17.386483 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:17.386771 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:17.387275 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:17.387916 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:17.388125 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:17.388171 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:17.390382 (+ 2211us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.158.65:54627: BlockingRecv error: recv got EOF from 127.5.158.65:54627 (error 108)
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":63,"threads_started":1}
Sep 25 07:26:17 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.66@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 07:26:17 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.66@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 07:26:17.510564 14951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:17.506121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42669 (local address 127.5.158.66:42739)
0925 07:26:17.506268 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:17.506278 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:17.506332 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:17.506445 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:17.506445 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:17.506642 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:17.506692 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:17.506694 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:17.507321 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:17.507322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:17.507829 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:17.507830 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:17.508155 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:17.508156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:17.508189 (+ 33us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:17.508591 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:17.508595 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:17.510473 (+ 1878us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.158.126:42669: Server oryx/127.5.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":94,"threads_started":1}
W20260925 07:26:17.510633 14950 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:26:17.506172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:42739 (local address 127.5.158.126:42669)
0925 07:26:17.506234 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:17.506236 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:17.506332 (+ 96us) server_negotiation.cc:407] Connection header received
0925 07:26:17.506383 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:17.506383 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:17.506393 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:17.506431 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:17.506759 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:17.507256 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:17.507894 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:17.508101 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:17.508146 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:17.510576 (+ 2430us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.158.66:42739: BlockingRecv error: recv got EOF from 127.5.158.66:42739 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Sep 25 07:26:17 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.67@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 07:26:17 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.67@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 07:26:17.631521 14952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:17.626631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42669 (local address 127.5.158.67:37119)
0925 07:26:17.626788 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:17.626830 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:17.626929 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:17.627060 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:17.627060 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:17.627228 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:17.627273 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:17.627274 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:17.627925 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:17.627926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:17.628528 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:17.628530 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:17.628907 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:17.628907 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:17.628973 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:17.629711 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:17.629716 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:17.631465 (+ 1749us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.158.126:42669: Server oryx/127.5.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":86,"threads_started":1}
W20260925 07:26:17.631635 14950 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:26:17.626707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:37119 (local address 127.5.158.126:42669)
0925 07:26:17.626784 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:17.626786 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:17.626911 (+ 125us) server_negotiation.cc:407] Connection header received
0925 07:26:17.626982 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:17.626983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:17.626998 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:17.627042 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:17.627349 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:17.627842 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:17.628615 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:17.628821 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:17.628905 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:17.631543 (+ 2638us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.158.67:37119: BlockingRecv error: recv got EOF from 127.5.158.67:37119 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Sep 25 07:26:18 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.65@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 07:26:18 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.65@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 07:26:18.395346 14953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:18.390888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42669 (local address 127.5.158.65:38413)
0925 07:26:18.391052 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:18.391063 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:18.391118 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:18.391256 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:18.391256 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:18.391411 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:18.391461 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:18.391462 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:18.392115 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:18.392115 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:18.392686 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:18.392688 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:18.393061 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:18.393062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:18.393154 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:18.393601 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:18.393605 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:18.395292 (+ 1687us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.158.126:42669: Server oryx/127.5.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":82,"threads_started":1}
W20260925 07:26:18.395457 14954 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:26:18.390987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:38413 (local address 127.5.158.126:42669)
0925 07:26:18.391091 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:18.391092 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:18.391103 (+ 11us) server_negotiation.cc:407] Connection header received
0925 07:26:18.391173 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:18.391174 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:18.391183 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:18.391240 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:18.391521 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:18.392054 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:18.392797 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:18.393000 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:18.393049 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:18.395378 (+ 2329us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.158.65:38413: BlockingRecv error: recv got EOF from 127.5.158.65:38413 (error 108)
Metrics: {"server-negotiator.queue_time_us":76,"thread_start_us":50,"threads_started":1}
Sep 25 07:26:18 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.66@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 07:26:18 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.66@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 07:26:18.515874 14955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:18.510986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42669 (local address 127.5.158.66:36725)
0925 07:26:18.511154 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:18.511171 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:18.511223 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:18.511344 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:18.511344 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:18.511501 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:18.511550 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:18.511552 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:18.512205 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:18.512206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:18.512740 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:18.512741 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:18.513064 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:18.513065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:18.513169 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:18.513745 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:18.513750 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:18.515787 (+ 2037us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.158.126:42669: Server oryx/127.5.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":119,"threads_started":1}
W20260925 07:26:18.515942 14954 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:26:18.511077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:36725 (local address 127.5.158.126:42669)
0925 07:26:18.511140 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:18.511142 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:18.511224 (+ 82us) server_negotiation.cc:407] Connection header received
0925 07:26:18.511265 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:18.511266 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:18.511276 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:18.511326 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:18.511606 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:18.512114 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:18.512813 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:18.513001 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:18.513047 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:18.515864 (+ 2817us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.158.66:36725: BlockingRecv error: recv got EOF from 127.5.158.66:36725 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Sep 25 07:26:18 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.67@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 07:26:18 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.67@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 07:26:18.636317 14956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:18.632019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42669 (local address 127.5.158.67:56183)
0925 07:26:18.632158 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:18.632170 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:18.632237 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:18.632361 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:18.632362 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:18.632504 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:18.632550 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:18.632551 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:18.633165 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:18.633166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:18.633674 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:18.633676 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:18.633963 (+ 287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:18.633964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:18.634000 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:18.634385 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:18.634389 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:18.636266 (+ 1877us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.158.126:42669: Server oryx/127.5.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":93,"threads_started":1}
W20260925 07:26:18.636420 14954 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:26:18.632097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:56183 (local address 127.5.158.126:42669)
0925 07:26:18.632140 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:18.632142 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:18.632222 (+ 80us) server_negotiation.cc:407] Connection header received
0925 07:26:18.632293 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:18.632293 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:18.632303 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:18.632342 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:18.632609 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:18.633068 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:18.633739 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:18.633905 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:18.633951 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:18.636351 (+ 2400us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.158.67:56183: BlockingRecv error: recv got EOF from 127.5.158.67:56183 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Sep 25 07:26:19 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.65@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 07:26:19 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.65@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 07:26:19.400041 14957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:19.395792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42669 (local address 127.5.158.65:35113)
0925 07:26:19.395921 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:19.395949 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:19.395994 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:19.396151 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:19.396152 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:19.396294 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:19.396343 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:19.396345 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:19.396979 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:19.396979 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:19.397585 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:19.397588 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:19.397900 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:19.397901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:19.397957 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:19.398368 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:19.398372 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:19.399978 (+ 1606us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.158.126:42669: Server oryx/127.5.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":74,"threads_started":1}
W20260925 07:26:19.400146 14958 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:26:19.395891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:35113 (local address 127.5.158.126:42669)
0925 07:26:19.396006 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:19.396007 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:19.396014 (+ 7us) server_negotiation.cc:407] Connection header received
0925 07:26:19.396045 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:19.396046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:19.396055 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:19.396138 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:19.396400 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:19.396905 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:19.397657 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:19.397840 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:19.397890 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:19.400070 (+ 2180us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.158.65:35113: BlockingRecv error: recv got EOF from 127.5.158.65:35113 (error 108)
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":65,"threads_started":1}
Sep 25 07:26:19 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.66@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 07:26:19 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.66@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 07:26:19.522043 14959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:19.516489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42669 (local address 127.5.158.66:44961)
0925 07:26:19.516613 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:19.516632 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:19.516683 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:19.517054 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:19.517056 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:19.517279 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:19.517335 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:19.517338 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:19.518582 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:19.518584 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:19.519386 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:19.519389 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:19.519872 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:19.519873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:19.519932 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:19.520347 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:19.520351 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:19.521989 (+ 1638us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.158.126:42669: Server oryx/127.5.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":70,"threads_started":1}
W20260925 07:26:19.522151 14958 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:26:19.516567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:44961 (local address 127.5.158.126:42669)
0925 07:26:19.516619 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:19.516621 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:19.516683 (+ 62us) server_negotiation.cc:407] Connection header received
0925 07:26:19.516760 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:19.516761 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:19.516776 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:19.516835 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:19.517442 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:19.518513 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:19.519467 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:19.519756 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:19.519859 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:19.522058 (+ 2199us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.158.66:44961: BlockingRecv error: recv got EOF from 127.5.158.66:44961 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Sep 25 07:26:19 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.67@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 07:26:19 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.67@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 07:26:19.641228 14960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:19.636786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42669 (local address 127.5.158.67:40855)
0925 07:26:19.636985 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:19.636994 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:19.637067 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:19.637250 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:19.637251 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:19.637414 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:19.637462 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:19.637464 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:19.638139 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:19.638140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:19.638623 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:19.638624 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:19.638974 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:19.638976 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:19.639038 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:19.639410 (+ 372us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:19.639415 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:19.641136 (+ 1721us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.158.126:42669: Server oryx/127.5.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":101,"threads_started":1}
W20260925 07:26:19.641240 14958 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:26:19.636838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:40855 (local address 127.5.158.126:42669)
0925 07:26:19.636923 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:19.636925 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:19.637054 (+ 129us) server_negotiation.cc:407] Connection header received
0925 07:26:19.637156 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:19.637157 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:19.637171 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:19.637244 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:19.637542 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:19.638044 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:19.638704 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:19.638886 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:19.638971 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:19.641186 (+ 2215us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.158.67:40855: BlockingRecv error: recv got EOF from 127.5.158.67:40855 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Sep 25 07:26:20 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.65@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 07:26:20 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.65@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 07:26:20.405467 14961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:20.400539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42669 (local address 127.5.158.65:55767)
0925 07:26:20.400683 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:20.400693 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:20.400749 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:20.400933 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:20.400934 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:20.401108 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:20.401157 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:20.401159 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:20.401992 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:20.401993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:20.402577 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:20.402580 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:20.402957 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:20.402958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:20.403027 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:20.403521 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:20.403533 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:20.405376 (+ 1843us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.158.126:42669: Server oryx/127.5.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":87,"threads_started":1}
W20260925 07:26:20.405512 14962 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:26:20.400661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:55767 (local address 127.5.158.126:42669)
0925 07:26:20.400801 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:20.400803 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:20.400818 (+ 15us) server_negotiation.cc:407] Connection header received
0925 07:26:20.400838 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:20.400839 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:20.400849 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:20.400909 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:20.401221 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:20.401909 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:20.402664 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:20.402879 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:20.402953 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:20.405446 (+ 2493us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.158.65:55767: BlockingRecv error: recv got EOF from 127.5.158.65:55767 (error 108)
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":87,"threads_started":1}
Sep 25 07:26:20 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.66@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 07:26:20 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.66@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 07:26:20.528602 14963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:20.522498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42669 (local address 127.5.158.66:60121)
0925 07:26:20.522596 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:20.522621 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:20.522680 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:20.522813 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:20.522814 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:20.523127 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:20.523204 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:20.523206 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:20.524160 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:20.524162 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:20.525130 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:20.525133 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:20.525806 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:20.525807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:20.525874 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:20.526319 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:20.526323 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:20.528487 (+ 2164us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.158.126:42669: Server oryx/127.5.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":81,"thread_start_us":54,"threads_started":1}
W20260925 07:26:20.528862 14962 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:26:20.522545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:60121 (local address 127.5.158.126:42669)
0925 07:26:20.522591 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:20.522593 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:20.522665 (+ 72us) server_negotiation.cc:407] Connection header received
0925 07:26:20.522736 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:20.522737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:20.522749 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:20.522982 (+ 233us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:20.523273 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:20.524061 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:20.525212 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:20.525505 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:20.525585 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:20.528798 (+ 3213us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.158.66:60121: BlockingRecv error: recv got EOF from 127.5.158.66:60121 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Sep 25 07:26:20 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.67@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 07:26:20 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.67@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 07:26:20.645558 14964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:20.641635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42669 (local address 127.5.158.67:43721)
0925 07:26:20.641754 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:20.641765 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:20.641824 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:20.641923 (+ 99us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:20.641924 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:20.642063 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:20.642166 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:20.642168 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:20.642711 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:20.642711 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:20.643157 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:20.643158 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:20.643422 (+ 264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:20.643423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:20.643455 (+ 32us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:20.643826 (+ 371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:20.643829 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:20.645471 (+ 1642us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.158.126:42669: Server oryx/127.5.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":69,"threads_started":1}
W20260925 07:26:20.645599 14962 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:26:20.641727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:43721 (local address 127.5.158.126:42669)
0925 07:26:20.641766 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:20.641768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:20.641810 (+ 42us) server_negotiation.cc:407] Connection header received
0925 07:26:20.641871 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:20.641872 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:20.641883 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:20.641918 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:20.642221 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:20.642651 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:20.643217 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:20.643370 (+ 153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:20.643410 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:20.645543 (+ 2133us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.158.67:43721: BlockingRecv error: recv got EOF from 127.5.158.67:43721 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
Sep 25 07:26:21 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.65@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 07:26:21 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.65@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 07:26:21.413949 14965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:21.405926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42669 (local address 127.5.158.65:44513)
0925 07:26:21.406063 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:21.406075 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:21.406151 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:21.406353 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:21.406355 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:21.406561 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:21.406623 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:21.406627 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:21.407347 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:21.407348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:21.407902 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:21.407904 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:21.408242 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:21.408243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:21.408282 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:21.412130 (+ 3848us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:21.412135 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:21.413855 (+ 1720us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.158.126:42669: Server oryx/127.5.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":49,"threads_started":1}
W20260925 07:26:21.414008 14966 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:26:21.406019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:44513 (local address 127.5.158.126:42669)
0925 07:26:21.406177 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:21.406179 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:21.406216 (+ 37us) server_negotiation.cc:407] Connection header received
0925 07:26:21.406255 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:21.406256 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:21.406268 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:21.406336 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:21.406723 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:21.407269 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:21.407980 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:21.408167 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:21.408224 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:21.413922 (+ 5698us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.158.65:44513: BlockingRecv error: recv got EOF from 127.5.158.65:44513 (error 108)
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":74,"threads_started":1}
Sep 25 07:26:21 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.66@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 07:26:21 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.66@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 07:26:21.533589 14967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:21.529209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42669 (local address 127.5.158.66:52947)
0925 07:26:21.529345 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:21.529357 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:21.529430 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:21.529544 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:21.529545 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:21.529717 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:21.529760 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:21.529762 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:21.530388 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:21.530389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:21.530889 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:21.530890 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:21.531231 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:21.531231 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:21.531311 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:21.531780 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:21.531784 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:21.533524 (+ 1740us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.158.126:42669: Server oryx/127.5.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":69,"threads_started":1}
W20260925 07:26:21.533684 14966 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:26:21.529279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:52947 (local address 127.5.158.126:42669)
0925 07:26:21.529348 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:21.529362 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:21.529406 (+ 44us) server_negotiation.cc:407] Connection header received
0925 07:26:21.529480 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:21.529481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:21.529494 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:21.529540 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:21.529836 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:21.530303 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:21.530971 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:21.531163 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:21.531232 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:21.533620 (+ 2388us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.158.66:52947: BlockingRecv error: recv got EOF from 127.5.158.66:52947 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Sep 25 07:26:21 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.67@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 07:26:21 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.67@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 07:26:21.650189 14968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:21.645999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42669 (local address 127.5.158.67:42011)
0925 07:26:21.646159 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:21.646171 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:21.646235 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:21.646343 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:21.646344 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:21.646482 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:21.646526 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:21.646527 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:21.647163 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:21.647163 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:21.647675 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:21.647676 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:21.647980 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:21.647981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:21.648020 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:21.648522 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:21.648526 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:21.650130 (+ 1604us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.158.126:42669: Server oryx/127.5.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":95,"threads_started":1}
W20260925 07:26:21.650300 14966 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:26:21.646141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:42011 (local address 127.5.158.126:42669)
0925 07:26:21.646181 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:21.646183 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:21.646213 (+ 30us) server_negotiation.cc:407] Connection header received
0925 07:26:21.646281 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:21.646281 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:21.646292 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:21.646328 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:21.646616 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:21.647095 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:21.647744 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:21.647920 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:21.647969 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:21.650210 (+ 2241us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.158.67:42011: BlockingRecv error: recv got EOF from 127.5.158.67:42011 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Sep 25 07:26:22 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.65@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 07:26:22 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.65@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 07:26:22.418545 14969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:22.414422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42669 (local address 127.5.158.65:59897)
0925 07:26:22.414578 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:22.414587 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:22.414633 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:22.414753 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:22.414754 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:22.414909 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:22.414954 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:22.414956 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:22.415525 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:22.415526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:22.416015 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:22.416016 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:22.416308 (+ 292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:22.416308 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:22.416353 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:22.416735 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:22.416738 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:22.418479 (+ 1741us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.158.126:42669: Server oryx/127.5.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":89,"threads_started":1}
W20260925 07:26:22.418665 14970 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:26:22.414503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:59897 (local address 127.5.158.126:42669)
0925 07:26:22.414627 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:22.414629 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:22.414636 (+ 7us) server_negotiation.cc:407] Connection header received
0925 07:26:22.414683 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:22.414683 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:22.414693 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:22.414737 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:22.415013 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:22.415466 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:22.416087 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:22.416253 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:22.416296 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:22.418566 (+ 2270us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.158.65:59897: BlockingRecv error: recv got EOF from 127.5.158.65:59897 (error 108)
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":58,"threads_started":1}
Sep 25 07:26:22 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.66@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 07:26:22 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.66@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 07:26:22.538475 14971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:22.534037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42669 (local address 127.5.158.66:48261)
0925 07:26:22.534186 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:22.534197 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:22.534261 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:22.534385 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:22.534385 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:22.534563 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:22.534615 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:22.534617 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:22.535293 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:22.535294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:22.535818 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:22.535819 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:22.536151 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:22.536152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:22.536209 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:22.536704 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:22.536707 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:22.538377 (+ 1670us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.158.126:42669: Server oryx/127.5.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":75,"threads_started":1}
W20260925 07:26:22.538514 14970 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:26:22.534119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:48261 (local address 127.5.158.126:42669)
0925 07:26:22.534172 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:22.534174 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:22.534244 (+ 70us) server_negotiation.cc:407] Connection header received
0925 07:26:22.534314 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:22.534315 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:22.534326 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:22.534380 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:22.534713 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:22.535211 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:22.535897 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:22.536075 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:22.536151 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:22.538457 (+ 2306us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.158.66:48261: BlockingRecv error: recv got EOF from 127.5.158.66:48261 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Sep 25 07:26:22 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.67@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 07:26:22 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.67@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 07:26:22.654757 14972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:22.650606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42669 (local address 127.5.158.67:39353)
0925 07:26:22.650757 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:22.650767 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:22.650821 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:22.650951 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:22.650952 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:22.651129 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:22.651171 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:22.651173 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:22.651791 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:22.651792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:22.652324 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:22.652325 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:22.652626 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:22.652626 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:22.652664 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:22.653175 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:22.653179 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:22.654698 (+ 1519us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.158.126:42669: Server oryx/127.5.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":88,"threads_started":1}
W20260925 07:26:22.654821 14970 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:26:22.650661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:39353 (local address 127.5.158.126:42669)
0925 07:26:22.650725 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:22.650728 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:22.650826 (+ 98us) server_negotiation.cc:407] Connection header received
0925 07:26:22.650865 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:22.650866 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:22.650878 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:22.650924 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:22.651235 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:22.651728 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:22.652388 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:22.652559 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:22.652605 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:22.654777 (+ 2172us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.158.67:39353: BlockingRecv error: recv got EOF from 127.5.158.67:39353 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Sep 25 07:26:23 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.65@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 07:26:23 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.65@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 07:26:23.423411 14973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:23.419129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42669 (local address 127.5.158.65:38875)
0925 07:26:23.419278 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:23.419288 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:23.419340 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:23.419473 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:23.419473 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:23.419642 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:23.419685 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:23.419687 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:23.420463 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:23.420463 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:23.420983 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:23.420984 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:23.421295 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:23.421296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:23.421327 (+ 31us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:23.421737 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:23.421742 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:23.423362 (+ 1620us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.158.126:42669: Server oryx/127.5.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":74,"threads_started":1}
W20260925 07:26:23.423532 14974 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:26:23.419221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:38875 (local address 127.5.158.126:42669)
0925 07:26:23.419343 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:23.419344 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:23.419370 (+ 26us) server_negotiation.cc:407] Connection header received
0925 07:26:23.419388 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:23.419389 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:23.419399 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:23.419456 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:23.419730 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:23.420404 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:23.421042 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:23.421234 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:23.421297 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:23.423438 (+ 2141us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.158.65:38875: BlockingRecv error: recv got EOF from 127.5.158.65:38875 (error 108)
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":67,"threads_started":1}
Sep 25 07:26:23 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.66@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 07:26:23 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.66@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 07:26:23.542997 14975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:23.538942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42669 (local address 127.5.158.66:53193)
0925 07:26:23.539063 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:23.539072 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:23.539131 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:23.539229 (+ 98us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:23.539229 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:23.539378 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:23.539425 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:23.539427 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:23.539976 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:23.539977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:23.540459 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:23.540460 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:23.540721 (+ 261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:23.540721 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:23.540756 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:23.541280 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:23.541285 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:23.542939 (+ 1654us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.158.126:42669: Server oryx/127.5.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":74,"threads_started":1}
W20260925 07:26:23.543133 14974 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:26:23.538999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:53193 (local address 127.5.158.126:42669)
0925 07:26:23.539048 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:23.539050 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:23.539118 (+ 68us) server_negotiation.cc:407] Connection header received
0925 07:26:23.539171 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:23.539172 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:23.539181 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:23.539224 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:23.539475 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:23.539918 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:23.540515 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:23.540669 (+ 154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:23.540710 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:23.543039 (+ 2329us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.158.66:53193: BlockingRecv error: recv got EOF from 127.5.158.66:53193 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Sep 25 07:26:23 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.67@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 07:26:23 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.67@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 07:26:23.659664 14976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:23.655296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42669 (local address 127.5.158.67:52717)
0925 07:26:23.655417 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:23.655427 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:23.655486 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:23.655598 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:23.655598 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:23.655743 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:23.655787 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:23.655788 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:23.656450 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:23.656451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:23.657007 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:23.657010 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:23.657395 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:23.657396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:23.657462 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:23.657920 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:23.657928 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:23.659577 (+ 1649us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.158.126:42669: Server oryx/127.5.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":76,"threads_started":1}
W20260925 07:26:23.659701 14974 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:26:23.655368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:52717 (local address 127.5.158.126:42669)
0925 07:26:23.655432 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:23.655434 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:23.655473 (+ 39us) server_negotiation.cc:407] Connection header received
0925 07:26:23.655525 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:23.655526 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:23.655537 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:23.655584 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:23.655841 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:23.656368 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:23.657124 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:23.657317 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:23.657394 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:23.659646 (+ 2252us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.158.67:52717: BlockingRecv error: recv got EOF from 127.5.158.67:52717 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 25 07:26:24 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.65@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 07:26:24 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.65@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 07:26:24.428409 14977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:24.423851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42669 (local address 127.5.158.65:37487)
0925 07:26:24.424032 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:24.424045 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:24.424109 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:24.424334 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:24.424335 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:24.424488 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:24.424531 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:24.424533 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:24.425160 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:24.425160 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:24.425713 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:24.425714 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:24.426064 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:24.426065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:24.426103 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:24.426576 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:24.426580 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:24.428333 (+ 1753us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.158.126:42669: Server oryx/127.5.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":68,"threads_started":1}
W20260925 07:26:24.428610 14978 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:26:24.424081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:37487 (local address 127.5.158.126:42669)
0925 07:26:24.424208 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:24.424215 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:24.424223 (+ 8us) server_negotiation.cc:407] Connection header received
0925 07:26:24.424249 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:24.424250 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:24.424260 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:24.424318 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:24.424595 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:24.425068 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:24.425801 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:24.426001 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:24.426055 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:24.428528 (+ 2473us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.158.65:37487: BlockingRecv error: recv got EOF from 127.5.158.65:37487 (error 108)
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":63,"threads_started":1}
Sep 25 07:26:24 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.66@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 07:26:24 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.66@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 07:26:24.547583 14979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:24.543397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42669 (local address 127.5.158.66:35511)
0925 07:26:24.543513 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:24.543523 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:24.543571 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:24.543687 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:24.543687 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:24.543825 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:24.543872 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:24.543873 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:24.544455 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:24.544456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:24.545028 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:24.545031 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:24.545394 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:24.545395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:24.545435 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:24.545835 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:24.545839 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:24.547530 (+ 1691us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.158.126:42669: Server oryx/127.5.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":70,"threads_started":1}
W20260925 07:26:24.547720 14978 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:26:24.543512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:35511 (local address 127.5.158.126:42669)
0925 07:26:24.543553 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:24.543555 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:24.543563 (+ 8us) server_negotiation.cc:407] Connection header received
0925 07:26:24.543625 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:24.543626 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:24.543635 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:24.543671 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:24.543933 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:24.544387 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:24.545131 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:24.545335 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:24.545386 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:24.547643 (+ 2257us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.158.66:35511: BlockingRecv error: recv got EOF from 127.5.158.66:35511 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Sep 25 07:26:24 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.67@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 07:26:24 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.67@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 07:26:24.664602 14980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:24.660222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42669 (local address 127.5.158.67:45977)
0925 07:26:24.660375 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:24.660385 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:24.660439 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:24.660556 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:24.660558 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:24.660717 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:24.660755 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:24.660757 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:24.661632 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:24.661633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:24.662114 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:24.662115 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:24.662422 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:24.662423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:24.662473 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:24.662828 (+ 355us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:24.662831 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:24.664521 (+ 1690us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.158.126:42669: Server oryx/127.5.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":102,"threads_started":1}
W20260925 07:26:24.664728 14978 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:26:24.660277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:45977 (local address 127.5.158.126:42669)
0925 07:26:24.660342 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:24.660344 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:24.660443 (+ 99us) server_negotiation.cc:407] Connection header received
0925 07:26:24.660482 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:24.660482 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:24.660495 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:24.660544 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:24.660837 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:24.661569 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:24.662174 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:24.662333 (+ 159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:24.662376 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:24.664618 (+ 2242us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.158.67:45977: BlockingRecv error: recv got EOF from 127.5.158.67:45977 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Sep 25 07:26:25 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.65@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 07:26:25 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.65@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 07:26:25.433442 14981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:25.428879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42669 (local address 127.5.158.65:34477)
0925 07:26:25.429044 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:25.429054 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:25.429260 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:25.429368 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:25.429368 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:25.429526 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:25.429570 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:25.429571 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:25.430100 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:25.430100 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:25.430558 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:25.430559 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:25.430846 (+ 287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:25.430847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:25.430885 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:25.431365 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:25.431369 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:25.433378 (+ 2009us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.158.126:42669: Server oryx/127.5.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":100,"threads_started":1}
W20260925 07:26:25.433578 14982 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:26:25.428993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:34477 (local address 127.5.158.126:42669)
0925 07:26:25.429128 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:25.429130 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:25.429259 (+ 129us) server_negotiation.cc:407] Connection header received
0925 07:26:25.429301 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:25.429302 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:25.429311 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:25.429362 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:25.429618 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:25.430043 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:25.430615 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:25.430774 (+ 159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:25.430833 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:25.433491 (+ 2658us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.158.65:34477: BlockingRecv error: recv got EOF from 127.5.158.65:34477 (error 108)
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":71,"threads_started":1}
Sep 25 07:26:25 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.66@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 07:26:25 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.66@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 07:26:25.552138 14983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:25.548012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42669 (local address 127.5.158.66:56079)
0925 07:26:25.548141 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:25.548160 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:25.548222 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:25.548382 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:25.548383 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:25.548530 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:25.548579 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:25.548580 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:25.549163 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:25.549164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:25.549649 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:25.549650 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:25.549922 (+ 272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:25.549923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:25.549978 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:25.550462 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:25.550466 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:25.552084 (+ 1618us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.158.126:42669: Server oryx/127.5.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":79,"threads_started":1}
W20260925 07:26:25.552263 14982 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:26:25.548074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:56079 (local address 127.5.158.126:42669)
0925 07:26:25.548142 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:25.548144 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:25.548215 (+ 71us) server_negotiation.cc:407] Connection header received
0925 07:26:25.548293 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:25.548294 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:25.548304 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:25.548354 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:25.548632 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:25.549051 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:25.549717 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:25.549873 (+ 156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:25.549915 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:25.552184 (+ 2269us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.158.66:56079: BlockingRecv error: recv got EOF from 127.5.158.66:56079 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Sep 25 07:26:25 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.67@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 07:26:25 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.67@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 07:26:25.669354 14984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:25.665042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42669 (local address 127.5.158.67:49887)
0925 07:26:25.665242 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:25.665253 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:25.665316 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:25.665450 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:25.665451 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:25.665619 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:25.665666 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:25.665667 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:25.666231 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:25.666232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:25.666701 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:25.666702 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:25.666996 (+ 294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:25.666997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:25.667033 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:25.667482 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:25.667487 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:25.669301 (+ 1814us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.158.126:42669: Server oryx/127.5.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":114,"threads_started":1}
W20260925 07:26:25.669466 14982 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:26:25.665107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:49887 (local address 127.5.158.126:42669)
0925 07:26:25.665155 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:25.665157 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:25.665315 (+ 158us) server_negotiation.cc:407] Connection header received
0925 07:26:25.665359 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:25.665360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:25.665372 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:25.665419 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:25.665724 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:25.666166 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:25.666771 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:25.666937 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:25.666981 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:25.669387 (+ 2406us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.158.67:49887: BlockingRecv error: recv got EOF from 127.5.158.67:49887 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Sep 25 07:26:26 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.65@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 07:26:26 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.65@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 07:26:26.443046 14985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:26.433925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42669 (local address 127.5.158.65:44593)
0925 07:26:26.434050 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:26.434059 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:26.434102 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:26.434271 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:26.434272 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:26.434413 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:26.434459 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:26.434460 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:26.435075 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:26.435076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:26.435512 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:26.435513 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:26.435823 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:26.435824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:26.435856 (+ 32us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:26.441236 (+ 5380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:26.441240 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:26.442989 (+ 1749us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.158.126:42669: Server oryx/127.5.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":72,"threads_started":1}
W20260925 07:26:26.443128 14986 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:26:26.434027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:44593 (local address 127.5.158.126:42669)
0925 07:26:26.434153 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:26.434155 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:26.434163 (+ 8us) server_negotiation.cc:407] Connection header received
0925 07:26:26.434183 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:26.434184 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:26.434196 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:26.434257 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:26.434523 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:26.435014 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:26.435580 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:26.435763 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:26.435818 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:26.443065 (+ 7247us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.158.65:44593: BlockingRecv error: recv got EOF from 127.5.158.65:44593 (error 108)
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":76,"threads_started":1}
Sep 25 07:26:26 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.66@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 07:26:26 dist-test-slave-1drq krb5kdc[13926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.158.66@KRBTEST.COM for oryx/127.5.158.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 07:26:26.556599 14987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:26.552670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:42669 (local address 127.5.158.66:32863)
0925 07:26:26.552793 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:26.552803 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:26.552860 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:26.552971 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:26.552972 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:26.553173 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:26.553219 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:26.553221 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:26.553840 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:26.553841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:26.554312 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:26.554313 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:26.554591 (+ 278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:26.554591 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:26.554628 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:26.555017 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:26.555021 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:26.556548 (+ 1527us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.158.126:42669: Server oryx/127.5.158.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":83,"threads_started":1}
W20260925 07:26:26.556753 14986 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:26:26.552733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:32863 (local address 127.5.158.126:42669)
0925 07:26:26.552796 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:26.552798 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:26.552845 (+ 47us) server_negotiation.cc:407] Connection header received
0925 07:26:26.552902 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:26.552903 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:26.552912 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:26.552957 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:26.553275 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:26.553777 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:26.554371 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:26.554536 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:26.554580 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:26.556648 (+ 2068us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.158.66:32863: BlockingRecv error: recv got EOF from 127.5.158.66:32863 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 07:26:26.590646 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 14509
I20260925 07:26:26.594901 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 14661
I20260925 07:26:26.599000 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 14794
I20260925 07:26:26.602763 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 14439
2026-09-25T07:26:26Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17340 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-25T07:26:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T07:26:26Z Disabled control of system clock
I20260925 07:26:26.628741 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:45117
--webserver_interface=127.5.158.126
--webserver_port=0
--builtin_ntp_servers=127.5.158.84:38139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.158.126:45117
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260925 07:26:26.699774 14991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:26:26.699991 14991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:26:26.700022 14991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:26:26.701131 14991 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:26:26.701192 14991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:26:26.701211 14991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:26:26.701232 14991 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:26:26.701260 14991 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:26:26.702504 14991 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:38139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.158.126:45117
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:45117
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:26:26.702745 14991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:26:26.703001 14991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:26:26.707825 14996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:26:26.707799 14997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:26:26.707780 14999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:26:26.707834 14991 server_base.cc:1034] running on GCE node
I20260925 07:26:26.708252 14991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:26:26.708544 14991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:26:26.709697 14991 hybrid_clock.cc:648] HybridClock initialized: now 1790321186709692 us; error 35 us; skew 500 ppm
I20260925 07:26:26.710793 14991 webserver.cc:460] Webserver started at http://127.5.158.126:33353/ using document root <none> and password file <none>
I20260925 07:26:26.711017 14991 fs_manager.cc:359] Metadata directory not provided
I20260925 07:26:26.711086 14991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:26:26.711230 14991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:26:26.712181 14991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/master-0/data/instance:
uuid: "61fa754ac8fa437881550d35021d9d3a"
format_stamp: "Formatted at 2026-09-25 07:26:26 on dist-test-slave-1drq"
server_key: "d43463a2db265753597a5c1ad327f70c"
server_key_iv: "3b3dad5a1e68e610ba11deca22552ffb"
server_key_version: "encryptionkey@0"
I20260925 07:26:26.712518 14991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/master-0/wal/instance:
uuid: "61fa754ac8fa437881550d35021d9d3a"
format_stamp: "Formatted at 2026-09-25 07:26:26 on dist-test-slave-1drq"
server_key: "d43463a2db265753597a5c1ad327f70c"
server_key_iv: "3b3dad5a1e68e610ba11deca22552ffb"
server_key_version: "encryptionkey@0"
I20260925 07:26:26.713806 14991 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 07:26:26.714488 15005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:26.714648 14991 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 07:26:26.714740 14991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "61fa754ac8fa437881550d35021d9d3a"
format_stamp: "Formatted at 2026-09-25 07:26:26 on dist-test-slave-1drq"
server_key: "d43463a2db265753597a5c1ad327f70c"
server_key_iv: "3b3dad5a1e68e610ba11deca22552ffb"
server_key_version: "encryptionkey@0"
I20260925 07:26:26.714815 14991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:26:26.731930 14991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:26:26.732160 14991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:26:26.732299 14991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:26:26.736076 14991 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.126:45117
I20260925 07:26:26.736104 15057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.126:45117 every 8 connection(s)
I20260925 07:26:26.736380 14991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:26:26.737012 15058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:26:26.739301 15058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61fa754ac8fa437881550d35021d9d3a: Bootstrap starting.
I20260925 07:26:26.739866 15058 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 61fa754ac8fa437881550d35021d9d3a: Neither blocks nor log segments found. Creating new log.
I20260925 07:26:26.740156 15058 log.cc:824] T 00000000000000000000000000000000 P 61fa754ac8fa437881550d35021d9d3a: Log is configured to *not* fsync() on all Append() calls
I20260925 07:26:26.740790 15058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61fa754ac8fa437881550d35021d9d3a: No bootstrap required, opened a new log
I20260925 07:26:26.742064 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 14991
I20260925 07:26:26.742134 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/master-0/wal/instance
I20260925 07:26:26.742314 5753 external_mini_cluster.cc:1428] Setting key fe1e4988f10c7d7973507630f90ddd26
I20260925 07:26:26.742414 15058 raft_consensus.cc:348] T 00000000000000000000000000000000 P 61fa754ac8fa437881550d35021d9d3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61fa754ac8fa437881550d35021d9d3a" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 45117 } }
I20260925 07:26:26.742591 15058 raft_consensus.cc:374] T 00000000000000000000000000000000 P 61fa754ac8fa437881550d35021d9d3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:26:26.742635 15058 raft_consensus.cc:729] T 00000000000000000000000000000000 P 61fa754ac8fa437881550d35021d9d3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61fa754ac8fa437881550d35021d9d3a, State: Initialized, Role: FOLLOWER
I20260925 07:26:26.742836 15058 consensus_queue.cc:260] T 00000000000000000000000000000000 P 61fa754ac8fa437881550d35021d9d3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61fa754ac8fa437881550d35021d9d3a" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 45117 } }
I20260925 07:26:26.742942 15058 raft_consensus.cc:388] T 00000000000000000000000000000000 P 61fa754ac8fa437881550d35021d9d3a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 07:26:26.742987 15058 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61fa754ac8fa437881550d35021d9d3a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 07:26:26.743047 15058 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61fa754ac8fa437881550d35021d9d3a [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:26:26.743518 15058 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61fa754ac8fa437881550d35021d9d3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61fa754ac8fa437881550d35021d9d3a" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 45117 } }
I20260925 07:26:26.743630 15058 leader_election.cc:302] T 00000000000000000000000000000000 P 61fa754ac8fa437881550d35021d9d3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 61fa754ac8fa437881550d35021d9d3a; no voters:
I20260925 07:26:26.744014 15058 leader_election.cc:288] T 00000000000000000000000000000000 P 61fa754ac8fa437881550d35021d9d3a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 07:26:26.744045 15063 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 61fa754ac8fa437881550d35021d9d3a [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:26:26.744168 15063 raft_consensus.cc:686] T 00000000000000000000000000000000 P 61fa754ac8fa437881550d35021d9d3a [term 1 LEADER]: Becoming Leader. State: Replica: 61fa754ac8fa437881550d35021d9d3a, State: Running, Role: LEADER
I20260925 07:26:26.744290 15058 sys_catalog.cc:564] T 00000000000000000000000000000000 P 61fa754ac8fa437881550d35021d9d3a [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:26:26.744329 15063 consensus_queue.cc:237] T 00000000000000000000000000000000 P 61fa754ac8fa437881550d35021d9d3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61fa754ac8fa437881550d35021d9d3a" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 45117 } }
I20260925 07:26:26.744776 15065 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61fa754ac8fa437881550d35021d9d3a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "61fa754ac8fa437881550d35021d9d3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61fa754ac8fa437881550d35021d9d3a" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 45117 } } }
I20260925 07:26:26.744866 15065 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61fa754ac8fa437881550d35021d9d3a [sys.catalog]: This master's current role is: LEADER
I20260925 07:26:26.745121 15064 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61fa754ac8fa437881550d35021d9d3a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 61fa754ac8fa437881550d35021d9d3a. Latest consensus state: current_term: 1 leader_uuid: "61fa754ac8fa437881550d35021d9d3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61fa754ac8fa437881550d35021d9d3a" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 45117 } } }
I20260925 07:26:26.745229 15064 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61fa754ac8fa437881550d35021d9d3a [sys.catalog]: This master's current role is: LEADER
I20260925 07:26:26.745406 15071 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 07:26:26.745890 15071 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 07:26:26.747905 15071 catalog_manager.cc:1347] Generated new cluster ID: fa3e7393d3b442378dddc1bdd9338248
I20260925 07:26:26.747987 15071 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 07:26:26.748472 15062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:26.742796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55966 (local address 127.5.158.126:45117)
0925 07:26:26.743070 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:26.743078 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:26.743106 (+ 28us) server_negotiation.cc:407] Connection header received
0925 07:26:26.743199 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:26.743204 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:26.743247 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:26.743372 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:26.743578 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:26.745203 (+ 1625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:26.746296 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:26.746853 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:26.746977 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:26.747675 (+ 698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:26:26.747679 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:26:26.747679 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:26:26.747880 (+ 201us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:26:26.747928 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:26:26.747942 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:26:26.748178 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0925 07:26:26.748209 (+ 31us) server_negotiation.cc:299] Negotiation successful
0925 07:26:26.748279 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":101,"threads_started":1}
I20260925 07:26:26.758920 15071 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 07:26:26.759511 15071 catalog_manager.cc:1504] Loading token signing keys...
I20260925 07:26:26.764366 15071 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 61fa754ac8fa437881550d35021d9d3a: Generated new TSK 0
I20260925 07:26:26.764496 15071 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 07:26:26.766481 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.65:0
--local_ip_for_outbound_sockets=127.5.158.65
--webserver_interface=127.5.158.65
--webserver_port=0
--tserver_master_addrs=127.5.158.126:45117
--builtin_ntp_servers=127.5.158.84:38139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260925 07:26:26.854717 15082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:26:26.854933 15082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:26:26.854966 15082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:26:26.856081 15082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:26:26.856150 15082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:26:26.856185 15082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.65
I20260925 07:26:26.857440 15082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:38139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.158.65
--webserver_port=0
--tserver_master_addrs=127.5.158.126:45117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15082
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.65
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:26:26.857678 15082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:26:26.857954 15082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:26:26.863456 15087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:26:26.863423 15090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:26:26.863425 15088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:26:26.863795 15082 server_base.cc:1034] running on GCE node
I20260925 07:26:26.863973 15082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:26:26.864283 15082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:26:26.865455 15082 hybrid_clock.cc:648] HybridClock initialized: now 1790321186865446 us; error 35 us; skew 500 ppm
I20260925 07:26:26.866679 15082 webserver.cc:460] Webserver started at http://127.5.158.65:35259/ using document root <none> and password file <none>
I20260925 07:26:26.866918 15082 fs_manager.cc:359] Metadata directory not provided
I20260925 07:26:26.867008 15082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:26:26.867156 15082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:26:26.868129 15082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-0/data/instance:
uuid: "64d6416cfda4475d9e6945171cd861f7"
format_stamp: "Formatted at 2026-09-25 07:26:26 on dist-test-slave-1drq"
server_key: "4ebdfce93182b306411cd024bdd15c3a"
server_key_iv: "716d962633f3ff525b2b8f7a39ea09a9"
server_key_version: "encryptionkey@0"
I20260925 07:26:26.868489 15082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-0/wal/instance:
uuid: "64d6416cfda4475d9e6945171cd861f7"
format_stamp: "Formatted at 2026-09-25 07:26:26 on dist-test-slave-1drq"
server_key: "4ebdfce93182b306411cd024bdd15c3a"
server_key_iv: "716d962633f3ff525b2b8f7a39ea09a9"
server_key_version: "encryptionkey@0"
I20260925 07:26:26.869745 15082 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 07:26:26.870424 15096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:26.870563 15082 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 07:26:26.870649 15082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-0/wal
uuid: "64d6416cfda4475d9e6945171cd861f7"
format_stamp: "Formatted at 2026-09-25 07:26:26 on dist-test-slave-1drq"
server_key: "4ebdfce93182b306411cd024bdd15c3a"
server_key_iv: "716d962633f3ff525b2b8f7a39ea09a9"
server_key_version: "encryptionkey@0"
I20260925 07:26:26.870708 15082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:26:26.897807 15082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:26:26.898077 15082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:26:26.898222 15082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:26:26.898463 15082 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:26:26.898738 15082 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:26:26.898769 15082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:26.898789 15082 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:26:26.898825 15082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:26.903919 15082 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.65:38031
I20260925 07:26:26.903985 15209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.65:38031 every 8 connection(s)
I20260925 07:26:26.904232 15082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
I20260925 07:26:26.908726 15062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:26.904855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:59947 (local address 127.5.158.126:45117)
0925 07:26:26.904905 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:26.904908 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:26.905200 (+ 292us) server_negotiation.cc:407] Connection header received
0925 07:26:26.905647 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:26.905649 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:26.905659 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:26.905706 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:26.906236 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:26.906665 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:26.907564 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:26.907739 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:26.907789 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:26.908520 (+ 731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:26:26.908521 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:26:26.908522 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:26:26.908525 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:26:26.908535 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:26:26.908540 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:26:26.908577 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0925 07:26:26.908652 (+ 75us) server_negotiation.cc:299] Negotiation successful
0925 07:26:26.908676 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 07:26:26.908887 15212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:26.904869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:45117 (local address 127.5.158.65:59947)
0925 07:26:26.905054 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:26.905111 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:26.905569 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:26.905776 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:26.905779 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:26.905786 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:26.906173 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:26.906176 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:26.906740 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:26.906741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:26.907491 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:26.907492 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:26.907833 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:26.907834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:26.907946 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:26.908388 (+ 442us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 07:26:26.908389 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:26.908413 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 07:26:26.908444 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:26:26.908588 (+ 144us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:26:26.908593 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:26:26.908594 (+ 1us) client_negotiation.cc:769] Sending connection context
0925 07:26:26.908642 (+ 48us) client_negotiation.cc:240] Negotiation successful
0925 07:26:26.908710 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":86,"threads_started":1}
I20260925 07:26:26.909364 15210 heartbeater.cc:347] Connected to a master server at 127.5.158.126:45117
I20260925 07:26:26.909458 15210 heartbeater.cc:464] Registering TS with master...
I20260925 07:26:26.909610 15210 heartbeater.cc:511] Master 127.5.158.126:45117 requested a full tablet report, sending...
I20260925 07:26:26.910027 15022 ts_manager.cc:194] Registered new tserver with Master: 64d6416cfda4475d9e6945171cd861f7 (127.5.158.65:38031)
I20260925 07:26:26.910563 15022 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.158.65:59947
I20260925 07:26:26.911801 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 15082
I20260925 07:26:26.911906 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-0/wal/instance
I20260925 07:26:26.912091 5753 external_mini_cluster.cc:1428] Setting key 6497d6c31ba8992c6b36fa0e97fb7610
I20260925 07:26:26.912851 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.66:0
--local_ip_for_outbound_sockets=127.5.158.66
--webserver_interface=127.5.158.66
--webserver_port=0
--tserver_master_addrs=127.5.158.126:45117
--builtin_ntp_servers=127.5.158.84:38139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260925 07:26:26.985522 15213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:26:26.985708 15213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:26:26.985728 15213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:26:26.986662 15213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:26:26.986701 15213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:26:26.986724 15213 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.66
I20260925 07:26:26.987932 15213 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:38139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.158.66
--webserver_port=0
--tserver_master_addrs=127.5.158.126:45117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15213
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.66
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:26:26.988108 15213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:26:26.988339 15213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:26:26.993052 15218 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:26:26.993203 15213 server_base.cc:1034] running on GCE node
W20260925 07:26:26.993052 15221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:26:26.993064 15219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:26:26.993506 15213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:26:26.993716 15213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:26:26.994864 15213 hybrid_clock.cc:648] HybridClock initialized: now 1790321186994851 us; error 41 us; skew 500 ppm
I20260925 07:26:26.996058 15213 webserver.cc:460] Webserver started at http://127.5.158.66:35971/ using document root <none> and password file <none>
I20260925 07:26:26.996275 15213 fs_manager.cc:359] Metadata directory not provided
I20260925 07:26:26.996345 15213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:26:26.996488 15213 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:26:26.997535 15213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-1/data/instance:
uuid: "68472a4162b34ec08db0f304134ecda8"
format_stamp: "Formatted at 2026-09-25 07:26:26 on dist-test-slave-1drq"
server_key: "c9eb879467f627a55e14dc7be7f0598d"
server_key_iv: "2dc4dae5951ff27be6f688491b435926"
server_key_version: "encryptionkey@0"
I20260925 07:26:26.997884 15213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-1/wal/instance:
uuid: "68472a4162b34ec08db0f304134ecda8"
format_stamp: "Formatted at 2026-09-25 07:26:26 on dist-test-slave-1drq"
server_key: "c9eb879467f627a55e14dc7be7f0598d"
server_key_iv: "2dc4dae5951ff27be6f688491b435926"
server_key_version: "encryptionkey@0"
I20260925 07:26:26.999078 15213 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 07:26:26.999680 15227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:26.999838 15213 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:26:26.999925 15213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-1/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-1/wal
uuid: "68472a4162b34ec08db0f304134ecda8"
format_stamp: "Formatted at 2026-09-25 07:26:26 on dist-test-slave-1drq"
server_key: "c9eb879467f627a55e14dc7be7f0598d"
server_key_iv: "2dc4dae5951ff27be6f688491b435926"
server_key_version: "encryptionkey@0"
I20260925 07:26:26.999997 15213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:26:27.013077 15213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:26:27.013363 15213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:26:27.013495 15213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:26:27.013707 15213 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:26:27.013999 15213 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:26:27.014071 15213 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:27.014104 15213 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:26:27.014132 15213 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:27.019842 15213 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.66:42381
I20260925 07:26:27.019871 15340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.66:42381 every 8 connection(s)
I20260925 07:26:27.020149 15213 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
I20260925 07:26:27.024905 15062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:27.020762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:44969 (local address 127.5.158.126:45117)
0925 07:26:27.020809 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:27.020810 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:27.020995 (+ 185us) server_negotiation.cc:407] Connection header received
0925 07:26:27.021505 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:27.021506 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:27.021523 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:27.021574 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:27.022264 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:27.022759 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:27.023695 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:27.023887 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:27.023957 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:27.024623 (+ 666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:26:27.024625 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:26:27.024625 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:26:27.024629 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:26:27.024644 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:26:27.024648 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:26:27.024710 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 07:26:27.024807 (+ 97us) server_negotiation.cc:299] Negotiation successful
0925 07:26:27.024836 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 07:26:27.025067 15343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:27.020743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:45117 (local address 127.5.158.66:44969)
0925 07:26:27.020894 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:27.020921 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:27.021406 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:27.021639 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:27.021642 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:27.021650 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:27.022171 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:27.022175 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:27.022864 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:27.022865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:27.023606 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:27.023608 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:27.023975 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:27.023976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:27.024093 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:27.024489 (+ 396us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 07:26:27.024490 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:27.024512 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 07:26:27.024530 (+ 18us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:26:27.024710 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:26:27.024716 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:26:27.024718 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 07:26:27.024807 (+ 89us) client_negotiation.cc:240] Negotiation successful
0925 07:26:27.024889 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":61,"threads_started":1}
I20260925 07:26:27.025530 15341 heartbeater.cc:347] Connected to a master server at 127.5.158.126:45117
I20260925 07:26:27.025594 15341 heartbeater.cc:464] Registering TS with master...
I20260925 07:26:27.025759 15341 heartbeater.cc:511] Master 127.5.158.126:45117 requested a full tablet report, sending...
I20260925 07:26:27.026147 15022 ts_manager.cc:194] Registered new tserver with Master: 68472a4162b34ec08db0f304134ecda8 (127.5.158.66:42381)
I20260925 07:26:27.026510 15022 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.158.66:44969
I20260925 07:26:27.026742 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 15213
I20260925 07:26:27.026858 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-1/wal/instance
I20260925 07:26:27.027040 5753 external_mini_cluster.cc:1428] Setting key e3c1adbe4ddc0d8f743ef651cdda73a7
I20260925 07:26:27.027874 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.67:0
--local_ip_for_outbound_sockets=127.5.158.67
--webserver_interface=127.5.158.67
--webserver_port=0
--tserver_master_addrs=127.5.158.126:45117
--builtin_ntp_servers=127.5.158.84:38139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260925 07:26:27.103610 15344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:26:27.103812 15344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:26:27.103853 15344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:26:27.104948 15344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:26:27.104987 15344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:26:27.105014 15344 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.67
I20260925 07:26:27.106243 15344 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:38139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.158.67
--webserver_port=0
--tserver_master_addrs=127.5.158.126:45117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15344
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.67
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:26:27.106410 15344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:26:27.106624 15344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:26:27.111177 15349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:26:27.111188 15350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:26:27.111346 15344 server_base.cc:1034] running on GCE node
W20260925 07:26:27.111188 15352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:26:27.111642 15344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:26:27.111879 15344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:26:27.113029 15344 hybrid_clock.cc:648] HybridClock initialized: now 1790321187113023 us; error 40 us; skew 500 ppm
I20260925 07:26:27.114044 15344 webserver.cc:460] Webserver started at http://127.5.158.67:45503/ using document root <none> and password file <none>
I20260925 07:26:27.114256 15344 fs_manager.cc:359] Metadata directory not provided
I20260925 07:26:27.114317 15344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:26:27.114452 15344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:26:27.115375 15344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-2/data/instance:
uuid: "3efc3c77694246d8ba6ed2b48885c665"
format_stamp: "Formatted at 2026-09-25 07:26:27 on dist-test-slave-1drq"
server_key: "40012e2ee4403797c32796b4977a2072"
server_key_iv: "e12586f81dfa101fc02a9ecc4b80d569"
server_key_version: "encryptionkey@0"
I20260925 07:26:27.115700 15344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-2/wal/instance:
uuid: "3efc3c77694246d8ba6ed2b48885c665"
format_stamp: "Formatted at 2026-09-25 07:26:27 on dist-test-slave-1drq"
server_key: "40012e2ee4403797c32796b4977a2072"
server_key_iv: "e12586f81dfa101fc02a9ecc4b80d569"
server_key_version: "encryptionkey@0"
I20260925 07:26:27.116876 15344 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 07:26:27.117660 15358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:27.117823 15344 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:27.117933 15344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-2/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-2/wal
uuid: "3efc3c77694246d8ba6ed2b48885c665"
format_stamp: "Formatted at 2026-09-25 07:26:27 on dist-test-slave-1drq"
server_key: "40012e2ee4403797c32796b4977a2072"
server_key_iv: "e12586f81dfa101fc02a9ecc4b80d569"
server_key_version: "encryptionkey@0"
I20260925 07:26:27.118013 15344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:26:27.127755 15344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:26:27.128003 15344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:26:27.128144 15344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:26:27.128351 15344 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:26:27.128623 15344 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:26:27.128670 15344 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:27.128703 15344 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:26:27.128726 15344 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:27.134055 15344 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.67:40099
I20260925 07:26:27.134092 15471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.67:40099 every 8 connection(s)
I20260925 07:26:27.134344 15344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
I20260925 07:26:27.138981 15062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:27.134871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:35935 (local address 127.5.158.126:45117)
0925 07:26:27.134907 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:27.134909 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:27.135095 (+ 186us) server_negotiation.cc:407] Connection header received
0925 07:26:27.135557 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:27.135558 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:27.135572 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:27.135615 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:27.136195 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:27.136658 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:27.137703 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:27.137906 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:27.137992 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:27.138701 (+ 709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:26:27.138702 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:26:27.138703 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:26:27.138707 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:26:27.138721 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:26:27.138726 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:26:27.138803 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 07:26:27.138883 (+ 80us) server_negotiation.cc:299] Negotiation successful
0925 07:26:27.138911 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260925 07:26:27.139173 15474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:27.134849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:45117 (local address 127.5.158.67:35935)
0925 07:26:27.134990 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:27.135016 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:27.135454 (+ 438us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:27.135682 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:27.135686 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:27.135695 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:27.136110 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:27.136113 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:27.136792 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:27.136793 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:27.137615 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:27.137617 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:27.138001 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:27.138002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:27.138125 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:27.138575 (+ 450us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 07:26:27.138576 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:27.138598 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 07:26:27.138614 (+ 16us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:26:27.138806 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:26:27.138811 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:26:27.138814 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 07:26:27.138884 (+ 70us) client_negotiation.cc:240] Negotiation successful
0925 07:26:27.138968 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":59,"threads_started":1}
I20260925 07:26:27.139642 15472 heartbeater.cc:347] Connected to a master server at 127.5.158.126:45117
I20260925 07:26:27.139706 15472 heartbeater.cc:464] Registering TS with master...
I20260925 07:26:27.139825 15472 heartbeater.cc:511] Master 127.5.158.126:45117 requested a full tablet report, sending...
I20260925 07:26:27.140111 15022 ts_manager.cc:194] Registered new tserver with Master: 3efc3c77694246d8ba6ed2b48885c665 (127.5.158.67:40099)
I20260925 07:26:27.140462 15022 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.158.67:35935
I20260925 07:26:27.142243 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 15344
I20260925 07:26:27.142326 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-2/wal/instance
I20260925 07:26:27.142509 5753 external_mini_cluster.cc:1428] Setting key 6a2b0404ce6a1dbde90dbc9ebd500a58
I20260925 07:26:27.143311 5753 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260925 07:26:27.144984 15062 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:26:27.144497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55968 (local address 127.5.158.126:45117)
0925 07:26:27.144528 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:27.144530 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:27.144653 (+ 123us) server_negotiation.cc:407] Connection header received
0925 07:26:27.144720 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:27.144720 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:27.144746 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:27.144819 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:27.144934 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55968: BlockingRecv error: recv got EOF from 127.0.0.1:55968 (error 108)
Metrics: {"server-negotiator.queue_time_us":14}
I20260925 07:26:27.145066 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 15082
I20260925 07:26:27.149610 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 15213
I20260925 07:26:27.153307 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 15344
I20260925 07:26:27.156828 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 14991
2026-09-25T07:26:27Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (553 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-25T07:26:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T07:26:27Z Disabled control of system clock
I20260925 07:26:27.182283 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:40895
--webserver_interface=127.5.158.126
--webserver_port=0
--builtin_ntp_servers=127.5.158.84:42299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.158.126:40895
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260925 07:26:27.255481 15483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:26:27.255661 15483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:26:27.255682 15483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:26:27.256603 15483 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:26:27.256636 15483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:26:27.256649 15483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:26:27.256667 15483 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:26:27.256680 15483 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:26:27.257948 15483 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:42299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.158.126:40895
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:40895
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15483
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:26:27.258126 15483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:26:27.258352 15483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:26:27.262465 15489 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:26:27.262590 15483 server_base.cc:1034] running on GCE node
W20260925 07:26:27.262447 15488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:26:27.262442 15491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:26:27.262909 15483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:26:27.263163 15483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:26:27.264331 15483 hybrid_clock.cc:648] HybridClock initialized: now 1790321187264324 us; error 31 us; skew 500 ppm
I20260925 07:26:27.265391 15483 webserver.cc:460] Webserver started at http://127.5.158.126:39925/ using document root <none> and password file <none>
I20260925 07:26:27.265594 15483 fs_manager.cc:359] Metadata directory not provided
I20260925 07:26:27.265653 15483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:26:27.265802 15483 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:26:27.266624 15483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/master-0/data/instance:
uuid: "a795faa58909486b81ece96a26ea8de7"
format_stamp: "Formatted at 2026-09-25 07:26:27 on dist-test-slave-1drq"
server_key: "557b3698ab521bb0f73b524b1a1ae31c"
server_key_iv: "c07cb17215660e79d68b9fd453da0fb3"
server_key_version: "encryptionkey@0"
I20260925 07:26:27.266966 15483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/master-0/wal/instance:
uuid: "a795faa58909486b81ece96a26ea8de7"
format_stamp: "Formatted at 2026-09-25 07:26:27 on dist-test-slave-1drq"
server_key: "557b3698ab521bb0f73b524b1a1ae31c"
server_key_iv: "c07cb17215660e79d68b9fd453da0fb3"
server_key_version: "encryptionkey@0"
I20260925 07:26:27.268195 15483 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 07:26:27.268852 15497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:27.268994 15483 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 07:26:27.269075 15483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "a795faa58909486b81ece96a26ea8de7"
format_stamp: "Formatted at 2026-09-25 07:26:27 on dist-test-slave-1drq"
server_key: "557b3698ab521bb0f73b524b1a1ae31c"
server_key_iv: "c07cb17215660e79d68b9fd453da0fb3"
server_key_version: "encryptionkey@0"
I20260925 07:26:27.269161 15483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:26:27.271871 15483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:26:27.272119 15483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:26:27.272241 15483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:26:27.275522 15483 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.126:40895
I20260925 07:26:27.275566 15549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.126:40895 every 8 connection(s)
I20260925 07:26:27.275866 15483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:26:27.276450 15550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:26:27.278918 15550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a795faa58909486b81ece96a26ea8de7: Bootstrap starting.
I20260925 07:26:27.279418 15550 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a795faa58909486b81ece96a26ea8de7: Neither blocks nor log segments found. Creating new log.
I20260925 07:26:27.279692 15550 log.cc:824] T 00000000000000000000000000000000 P a795faa58909486b81ece96a26ea8de7: Log is configured to *not* fsync() on all Append() calls
I20260925 07:26:27.280263 15550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a795faa58909486b81ece96a26ea8de7: No bootstrap required, opened a new log
I20260925 07:26:27.281651 15550 raft_consensus.cc:348] T 00000000000000000000000000000000 P a795faa58909486b81ece96a26ea8de7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a795faa58909486b81ece96a26ea8de7" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 40895 } }
I20260925 07:26:27.281767 15550 raft_consensus.cc:374] T 00000000000000000000000000000000 P a795faa58909486b81ece96a26ea8de7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:26:27.281809 15550 raft_consensus.cc:729] T 00000000000000000000000000000000 P a795faa58909486b81ece96a26ea8de7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a795faa58909486b81ece96a26ea8de7, State: Initialized, Role: FOLLOWER
I20260925 07:26:27.281952 15550 consensus_queue.cc:260] T 00000000000000000000000000000000 P a795faa58909486b81ece96a26ea8de7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a795faa58909486b81ece96a26ea8de7" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 40895 } }
I20260925 07:26:27.282024 15550 raft_consensus.cc:388] T 00000000000000000000000000000000 P a795faa58909486b81ece96a26ea8de7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 07:26:27.282094 15550 raft_consensus.cc:482] T 00000000000000000000000000000000 P a795faa58909486b81ece96a26ea8de7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 07:26:27.282140 15550 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a795faa58909486b81ece96a26ea8de7 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:26:27.282665 15550 raft_consensus.cc:504] T 00000000000000000000000000000000 P a795faa58909486b81ece96a26ea8de7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a795faa58909486b81ece96a26ea8de7" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 40895 } }
I20260925 07:26:27.282748 15550 leader_election.cc:302] T 00000000000000000000000000000000 P a795faa58909486b81ece96a26ea8de7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a795faa58909486b81ece96a26ea8de7; no voters:
I20260925 07:26:27.282909 15550 leader_election.cc:288] T 00000000000000000000000000000000 P a795faa58909486b81ece96a26ea8de7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 07:26:27.282951 15553 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a795faa58909486b81ece96a26ea8de7 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:26:27.283115 15553 raft_consensus.cc:686] T 00000000000000000000000000000000 P a795faa58909486b81ece96a26ea8de7 [term 1 LEADER]: Becoming Leader. State: Replica: a795faa58909486b81ece96a26ea8de7, State: Running, Role: LEADER
I20260925 07:26:27.283146 15550 sys_catalog.cc:564] T 00000000000000000000000000000000 P a795faa58909486b81ece96a26ea8de7 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:26:27.283218 15553 consensus_queue.cc:237] T 00000000000000000000000000000000 P a795faa58909486b81ece96a26ea8de7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a795faa58909486b81ece96a26ea8de7" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 40895 } }
I20260925 07:26:27.283607 15555 sys_catalog.cc:455] T 00000000000000000000000000000000 P a795faa58909486b81ece96a26ea8de7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a795faa58909486b81ece96a26ea8de7. Latest consensus state: current_term: 1 leader_uuid: "a795faa58909486b81ece96a26ea8de7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a795faa58909486b81ece96a26ea8de7" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 40895 } } }
I20260925 07:26:27.283696 15555 sys_catalog.cc:458] T 00000000000000000000000000000000 P a795faa58909486b81ece96a26ea8de7 [sys.catalog]: This master's current role is: LEADER
I20260925 07:26:27.283769 15554 sys_catalog.cc:455] T 00000000000000000000000000000000 P a795faa58909486b81ece96a26ea8de7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a795faa58909486b81ece96a26ea8de7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a795faa58909486b81ece96a26ea8de7" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 40895 } } }
I20260925 07:26:27.283819 15554 sys_catalog.cc:458] T 00000000000000000000000000000000 P a795faa58909486b81ece96a26ea8de7 [sys.catalog]: This master's current role is: LEADER
I20260925 07:26:27.283937 15562 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 07:26:27.284363 15562 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 07:26:27.285780 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 15483
I20260925 07:26:27.285888 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/master-0/wal/instance
I20260925 07:26:27.285959 15562 catalog_manager.cc:1347] Generated new cluster ID: e2a1be1b80d3414cbe9a61ab366bf73c
I20260925 07:26:27.285996 15562 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 07:26:27.286080 5753 external_mini_cluster.cc:1428] Setting key 7f511cb28178319add1178613030c936
I20260925 07:26:27.290714 15573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:27.286525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53244 (local address 127.5.158.126:40895)
0925 07:26:27.286687 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:27.286695 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:27.289304 (+ 2609us) server_negotiation.cc:407] Connection header received
0925 07:26:27.289428 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:27.289434 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:27.289471 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:27.289587 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:27.290328 (+ 741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:26:27.290331 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:26:27.290332 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:26:27.290335 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:26:27.290371 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:26:27.290382 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:26:27.290437 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 07:26:27.290530 (+ 93us) server_negotiation.cc:299] Negotiation successful
0925 07:26:27.290575 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":66,"threads_started":1}
I20260925 07:26:27.307847 15562 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 07:26:27.308027 15562 catalog_manager.cc:1504] Loading token signing keys...
I20260925 07:26:27.313512 15562 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a795faa58909486b81ece96a26ea8de7: Generated new TSK 0
I20260925 07:26:27.313663 15562 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 07:26:27.315299 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.65:0
--local_ip_for_outbound_sockets=127.5.158.65
--webserver_interface=127.5.158.65
--webserver_port=0
--tserver_master_addrs=127.5.158.126:40895
--builtin_ntp_servers=127.5.158.84:42299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260925 07:26:27.397028 15574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:26:27.397280 15574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:26:27.397305 15574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:26:27.398473 15574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:26:27.398521 15574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:26:27.398545 15574 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.65
I20260925 07:26:27.399751 15574 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:42299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.158.65
--webserver_port=0
--tserver_master_addrs=127.5.158.126:40895
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15574
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.65
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:26:27.399953 15574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:26:27.400238 15574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 07:26:27.404974 15574 server_base.cc:1034] running on GCE node
W20260925 07:26:27.404927 15579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:26:27.405079 15580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:26:27.404937 15582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:26:27.405406 15574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:26:27.405643 15574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:26:27.406826 15574 hybrid_clock.cc:648] HybridClock initialized: now 1790321187406811 us; error 30 us; skew 500 ppm
I20260925 07:26:27.408071 15574 webserver.cc:460] Webserver started at http://127.5.158.65:43621/ using document root <none> and password file <none>
I20260925 07:26:27.408325 15574 fs_manager.cc:359] Metadata directory not provided
I20260925 07:26:27.408397 15574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:26:27.408530 15574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:26:27.409582 15574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-0/data/instance:
uuid: "a1789ce3f0f84ab0861d04cdca1ba42c"
format_stamp: "Formatted at 2026-09-25 07:26:27 on dist-test-slave-1drq"
server_key: "0eb2bea9e889b634343a4d3dbce50194"
server_key_iv: "e6f6e3df28dd2d6effd1f0d6a8ae6bef"
server_key_version: "encryptionkey@0"
I20260925 07:26:27.409921 15574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-0/wal/instance:
uuid: "a1789ce3f0f84ab0861d04cdca1ba42c"
format_stamp: "Formatted at 2026-09-25 07:26:27 on dist-test-slave-1drq"
server_key: "0eb2bea9e889b634343a4d3dbce50194"
server_key_iv: "e6f6e3df28dd2d6effd1f0d6a8ae6bef"
server_key_version: "encryptionkey@0"
I20260925 07:26:27.411126 15574 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 07:26:27.411775 15588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:27.411962 15574 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:26:27.412047 15574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-0/wal
uuid: "a1789ce3f0f84ab0861d04cdca1ba42c"
format_stamp: "Formatted at 2026-09-25 07:26:27 on dist-test-slave-1drq"
server_key: "0eb2bea9e889b634343a4d3dbce50194"
server_key_iv: "e6f6e3df28dd2d6effd1f0d6a8ae6bef"
server_key_version: "encryptionkey@0"
I20260925 07:26:27.412122 15574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:26:27.415059 15574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:26:27.415319 15574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:26:27.415450 15574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:26:27.415627 15574 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:26:27.415899 15574 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:26:27.415947 15574 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:27.415982 15574 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:26:27.416006 15574 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:27.421147 15574 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.65:44363
I20260925 07:26:27.421190 15701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.65:44363 every 8 connection(s)
I20260925 07:26:27.421460 15574 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
I20260925 07:26:27.423807 15573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:27.422061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:51045 (local address 127.5.158.126:40895)
0925 07:26:27.422104 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:27.422106 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:27.422338 (+ 232us) server_negotiation.cc:407] Connection header received
0925 07:26:27.422705 (+ 367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:27.422706 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:27.422711 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:27.422744 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:27.423636 (+ 892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:26:27.423637 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:26:27.423638 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:26:27.423641 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:26:27.423654 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:26:27.423657 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:26:27.423694 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0925 07:26:27.423753 (+ 59us) server_negotiation.cc:299] Negotiation successful
0925 07:26:27.423764 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 07:26:27.423991 15704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:27.422061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:40895 (local address 127.5.158.65:51045)
0925 07:26:27.422225 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:27.422253 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:27.422621 (+ 368us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:27.422794 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:27.422797 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:27.422804 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:27.423520 (+ 716us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 07:26:27.423520 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:27.423541 (+ 21us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 07:26:27.423564 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:26:27.423701 (+ 137us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:26:27.423703 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:26:27.423705 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 07:26:27.423758 (+ 53us) client_negotiation.cc:240] Negotiation successful
0925 07:26:27.423795 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":79,"threads_started":1}
I20260925 07:26:27.424540 15702 heartbeater.cc:347] Connected to a master server at 127.5.158.126:40895
I20260925 07:26:27.424609 15702 heartbeater.cc:464] Registering TS with master...
I20260925 07:26:27.424762 15702 heartbeater.cc:511] Master 127.5.158.126:40895 requested a full tablet report, sending...
I20260925 07:26:27.425213 15514 ts_manager.cc:194] Registered new tserver with Master: a1789ce3f0f84ab0861d04cdca1ba42c (127.5.158.65:44363)
I20260925 07:26:27.429925 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 15574
I20260925 07:26:27.430018 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-0/wal/instance
I20260925 07:26:27.430194 5753 external_mini_cluster.cc:1428] Setting key 24989483c2a39c1e1e10671796cf2bbe
I20260925 07:26:27.430938 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.66:0
--local_ip_for_outbound_sockets=127.5.158.66
--webserver_interface=127.5.158.66
--webserver_port=0
--tserver_master_addrs=127.5.158.126:40895
--builtin_ntp_servers=127.5.158.84:42299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260925 07:26:27.503196 15705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:26:27.503389 15705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:26:27.503424 15705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:26:27.504542 15705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:26:27.504604 15705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:26:27.504648 15705 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.66
I20260925 07:26:27.505944 15705 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:42299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.158.66
--webserver_port=0
--tserver_master_addrs=127.5.158.126:40895
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15705
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.66
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:26:27.506160 15705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:26:27.506428 15705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:26:27.511037 15713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:26:27.510993 15710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:26:27.511000 15711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:26:27.511171 15705 server_base.cc:1034] running on GCE node
I20260925 07:26:27.511459 15705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:26:27.511636 15705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:26:27.512790 15705 hybrid_clock.cc:648] HybridClock initialized: now 1790321187512781 us; error 29 us; skew 500 ppm
I20260925 07:26:27.514140 15705 webserver.cc:460] Webserver started at http://127.5.158.66:33425/ using document root <none> and password file <none>
I20260925 07:26:27.514379 15705 fs_manager.cc:359] Metadata directory not provided
I20260925 07:26:27.514446 15705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:26:27.514585 15705 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:26:27.515645 15705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-1/data/instance:
uuid: "96c9d9b2bd2c4dcb8ac04878f104e729"
format_stamp: "Formatted at 2026-09-25 07:26:27 on dist-test-slave-1drq"
server_key: "6b6a4e07038c54cf0e8b13af0d4c5b94"
server_key_iv: "f8484e668778a371cbff820f3a2301b3"
server_key_version: "encryptionkey@0"
I20260925 07:26:27.515923 15705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-1/wal/instance:
uuid: "96c9d9b2bd2c4dcb8ac04878f104e729"
format_stamp: "Formatted at 2026-09-25 07:26:27 on dist-test-slave-1drq"
server_key: "6b6a4e07038c54cf0e8b13af0d4c5b94"
server_key_iv: "f8484e668778a371cbff820f3a2301b3"
server_key_version: "encryptionkey@0"
I20260925 07:26:27.517061 15705 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 07:26:27.517822 15719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:27.517995 15705 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:26:27.518090 15705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-1/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-1/wal
uuid: "96c9d9b2bd2c4dcb8ac04878f104e729"
format_stamp: "Formatted at 2026-09-25 07:26:27 on dist-test-slave-1drq"
server_key: "6b6a4e07038c54cf0e8b13af0d4c5b94"
server_key_iv: "f8484e668778a371cbff820f3a2301b3"
server_key_version: "encryptionkey@0"
I20260925 07:26:27.518170 15705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:26:27.521174 15705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:26:27.521417 15705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:26:27.521539 15705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:26:27.521780 15705 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:26:27.522065 15705 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:26:27.522112 15705 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:27.522147 15705 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:26:27.522171 15705 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:27.527797 15705 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.66:38709
I20260925 07:26:27.527840 15832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.66:38709 every 8 connection(s)
I20260925 07:26:27.528111 15705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
I20260925 07:26:27.530701 15573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:27.528735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:42823 (local address 127.5.158.126:40895)
0925 07:26:27.528777 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:27.528779 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:27.528984 (+ 205us) server_negotiation.cc:407] Connection header received
0925 07:26:27.529474 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:27.529475 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:27.529479 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:27.529548 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:27.530542 (+ 994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:26:27.530543 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:26:27.530543 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:26:27.530546 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:26:27.530557 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:26:27.530561 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:26:27.530598 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0925 07:26:27.530639 (+ 41us) server_negotiation.cc:299] Negotiation successful
0925 07:26:27.530649 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 07:26:27.530866 15835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:27.528713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:40895 (local address 127.5.158.66:42823)
0925 07:26:27.528884 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:27.528923 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:27.529375 (+ 452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:27.529593 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:27.529596 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:27.529603 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:27.530429 (+ 826us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 07:26:27.530430 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:27.530452 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 07:26:27.530476 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:26:27.530598 (+ 122us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:26:27.530600 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:26:27.530603 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 07:26:27.530646 (+ 43us) client_negotiation.cc:240] Negotiation successful
0925 07:26:27.530690 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":89,"threads_started":1}
I20260925 07:26:27.531333 15833 heartbeater.cc:347] Connected to a master server at 127.5.158.126:40895
I20260925 07:26:27.531417 15833 heartbeater.cc:464] Registering TS with master...
I20260925 07:26:27.531560 15833 heartbeater.cc:511] Master 127.5.158.126:40895 requested a full tablet report, sending...
I20260925 07:26:27.531942 15514 ts_manager.cc:194] Registered new tserver with Master: 96c9d9b2bd2c4dcb8ac04878f104e729 (127.5.158.66:38709)
I20260925 07:26:27.534852 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 15705
I20260925 07:26:27.534948 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-1/wal/instance
I20260925 07:26:27.535121 5753 external_mini_cluster.cc:1428] Setting key 4140642d29a67ee524a13985276671be
I20260925 07:26:27.536001 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.67:0
--local_ip_for_outbound_sockets=127.5.158.67
--webserver_interface=127.5.158.67
--webserver_port=0
--tserver_master_addrs=127.5.158.126:40895
--builtin_ntp_servers=127.5.158.84:42299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260925 07:26:27.610661 15836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:26:27.610852 15836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:26:27.610895 15836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:26:27.612011 15836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:26:27.612071 15836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:26:27.612114 15836 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.67
I20260925 07:26:27.613416 15836 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:42299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.158.67
--webserver_port=0
--tserver_master_addrs=127.5.158.126:40895
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15836
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.67
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:26:27.613634 15836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:26:27.613881 15836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:26:27.618360 15844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:26:27.618386 15841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:26:27.618337 15842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:26:27.618508 15836 server_base.cc:1034] running on GCE node
I20260925 07:26:27.618731 15836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:26:27.618932 15836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:26:27.620066 15836 hybrid_clock.cc:648] HybridClock initialized: now 1790321187620044 us; error 35 us; skew 500 ppm
I20260925 07:26:27.621376 15836 webserver.cc:460] Webserver started at http://127.5.158.67:43405/ using document root <none> and password file <none>
I20260925 07:26:27.621608 15836 fs_manager.cc:359] Metadata directory not provided
I20260925 07:26:27.621644 15836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:26:27.621760 15836 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:26:27.622610 15836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-2/data/instance:
uuid: "f5836e4a4d904f7296dd14d3ddc64b65"
format_stamp: "Formatted at 2026-09-25 07:26:27 on dist-test-slave-1drq"
server_key: "ec519f6a631cb7268f9df441b2fc931e"
server_key_iv: "7b8d6e41bf50c1a02d3a58de5202f051"
server_key_version: "encryptionkey@0"
I20260925 07:26:27.622890 15836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-2/wal/instance:
uuid: "f5836e4a4d904f7296dd14d3ddc64b65"
format_stamp: "Formatted at 2026-09-25 07:26:27 on dist-test-slave-1drq"
server_key: "ec519f6a631cb7268f9df441b2fc931e"
server_key_iv: "7b8d6e41bf50c1a02d3a58de5202f051"
server_key_version: "encryptionkey@0"
I20260925 07:26:27.623989 15836 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 07:26:27.624620 15850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:27.624820 15836 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:26:27.624907 15836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-2/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-2/wal
uuid: "f5836e4a4d904f7296dd14d3ddc64b65"
format_stamp: "Formatted at 2026-09-25 07:26:27 on dist-test-slave-1drq"
server_key: "ec519f6a631cb7268f9df441b2fc931e"
server_key_iv: "7b8d6e41bf50c1a02d3a58de5202f051"
server_key_version: "encryptionkey@0"
I20260925 07:26:27.624979 15836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:26:27.627925 15836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:26:27.628157 15836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:26:27.628294 15836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:26:27.628515 15836 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:26:27.628772 15836 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:26:27.628816 15836 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:27.628850 15836 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:26:27.628875 15836 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:27.633736 15836 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.67:43421
I20260925 07:26:27.633790 15963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.67:43421 every 8 connection(s)
I20260925 07:26:27.634052 15836 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
I20260925 07:26:27.636358 15573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:27.634636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:57679 (local address 127.5.158.126:40895)
0925 07:26:27.634695 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:27.634696 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:27.634882 (+ 186us) server_negotiation.cc:407] Connection header received
0925 07:26:27.635212 (+ 330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:27.635213 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:27.635217 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:27.635265 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:27.636161 (+ 896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:26:27.636162 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:26:27.636163 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:26:27.636165 (+ 2us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:26:27.636193 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:26:27.636197 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:26:27.636247 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0925 07:26:27.636297 (+ 50us) server_negotiation.cc:299] Negotiation successful
0925 07:26:27.636308 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 07:26:27.636544 15966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:27.634627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:40895 (local address 127.5.158.67:57679)
0925 07:26:27.634775 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:27.634807 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:27.635140 (+ 333us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:27.635303 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:27.635306 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:27.635313 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:27.636045 (+ 732us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 07:26:27.636045 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:27.636067 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 07:26:27.636089 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:26:27.636247 (+ 158us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:26:27.636250 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:26:27.636251 (+ 1us) client_negotiation.cc:769] Sending connection context
0925 07:26:27.636303 (+ 52us) client_negotiation.cc:240] Negotiation successful
0925 07:26:27.636342 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":75,"threads_started":1}
I20260925 07:26:27.636914 15964 heartbeater.cc:347] Connected to a master server at 127.5.158.126:40895
I20260925 07:26:27.636986 15964 heartbeater.cc:464] Registering TS with master...
I20260925 07:26:27.637069 15964 heartbeater.cc:511] Master 127.5.158.126:40895 requested a full tablet report, sending...
I20260925 07:26:27.637430 15514 ts_manager.cc:194] Registered new tserver with Master: f5836e4a4d904f7296dd14d3ddc64b65 (127.5.158.67:43421)
I20260925 07:26:27.640303 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 15836
I20260925 07:26:27.640400 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790321110323796-5753-0/minicluster-data/ts-2/wal/instance
I20260925 07:26:27.640573 5753 external_mini_cluster.cc:1428] Setting key c67bb54049369d0ca5b7de6b98d6b934
I20260925 07:26:27.641351 5753 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260925 07:26:27.642974 15573 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:26:27.642532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38236 (local address 127.5.158.126:40895)
0925 07:26:27.642602 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:27.642604 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:27.642689 (+ 85us) server_negotiation.cc:407] Connection header received
0925 07:26:27.642754 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:27.642755 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:27.642759 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:27.642807 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:27.642923 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38236: BlockingRecv error: recv got EOF from 127.0.0.1:38236 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 07:26:27.643065 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 15574
I20260925 07:26:27.647732 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 15705
I20260925 07:26:27.651350 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 15836
I20260925 07:26:27.655386 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 15483
2026-09-25T07:26:27Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (498 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 07:26:27 dist-test-slave-1drq krb5kdc[15974](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 07:26:27 dist-test-slave-1drq krb5kdc[15974](info): set up 2 sockets
Sep 25 07:26:27 dist-test-slave-1drq krb5kdc[15974](info): commencing operation
krb5kdc: starting...
W20260925 07:26:28.609469 5753 mini_kdc.cc:121] Time spent starting KDC: real 0.935s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Sep 25 07:26:28 dist-test-slave-1drq krb5kdc[15974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321188, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T07:26:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T07:26:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
I20260925 07:26:28.709864 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:45137
--webserver_interface=127.5.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:35759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.158.126:45137
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:26:28.781445 15987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:26:28.781641 15987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:26:28.781673 15987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:26:28.782732 15987 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:26:28.782809 15987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:26:28.782840 15987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:26:28.782907 15987 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:26:28.782931 15987 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:26:28.784330 15987 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:35759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.158.126:45137
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:45137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15987
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:26:28.784600 15987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:26:28.784797 15987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:26:28.789148 15992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:26:28.789126 15995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:26:28.789127 15993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:26:28.789219 15987 server_base.cc:1034] running on GCE node
I20260925 07:26:28.789516 15987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:26:28.789784 15987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:26:28.790936 15987 hybrid_clock.cc:648] HybridClock initialized: now 1790321188790932 us; error 31 us; skew 500 ppm
Sep 25 07:26:28 dist-test-slave-1drq krb5kdc[15974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321188, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:26:28.793123 15987 init.cc:377] Logged in from keytab as kudu/127.5.158.126@KRBTEST.COM (short username kudu)
I20260925 07:26:28.793476 15987 webserver.cc:460] Webserver started at http://127.5.158.126:42177/ using document root <none> and password file <none>
I20260925 07:26:28.793672 15987 fs_manager.cc:359] Metadata directory not provided
I20260925 07:26:28.793728 15987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:26:28.793859 15987 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:26:28.794708 15987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/master-0/data/instance:
uuid: "51e2568956264fd9944bb4f1ac73c90d"
format_stamp: "Formatted at 2026-09-25 07:26:28 on dist-test-slave-1drq"
server_key: "d1c159907f6c66a4ad53753ffc1764bc"
server_key_iv: "0f179b3462c2df25da9635867e504ae8"
server_key_version: "encryptionkey@0"
I20260925 07:26:28.795022 15987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/master-0/wal/instance:
uuid: "51e2568956264fd9944bb4f1ac73c90d"
format_stamp: "Formatted at 2026-09-25 07:26:28 on dist-test-slave-1drq"
server_key: "d1c159907f6c66a4ad53753ffc1764bc"
server_key_iv: "0f179b3462c2df25da9635867e504ae8"
server_key_version: "encryptionkey@0"
I20260925 07:26:28.796113 15987 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 07:26:28.796692 16002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:28.796826 15987 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:26:28.796902 15987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "51e2568956264fd9944bb4f1ac73c90d"
format_stamp: "Formatted at 2026-09-25 07:26:28 on dist-test-slave-1drq"
server_key: "d1c159907f6c66a4ad53753ffc1764bc"
server_key_iv: "0f179b3462c2df25da9635867e504ae8"
server_key_version: "encryptionkey@0"
I20260925 07:26:28.796990 15987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:26:28.833649 15987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:26:28.835708 15987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:26:28.835855 15987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:26:28.839462 15987 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.126:45137
I20260925 07:26:28.839493 16054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.126:45137 every 8 connection(s)
I20260925 07:26:28.839783 15987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:26:28.840600 16055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:26:28.842592 16055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51e2568956264fd9944bb4f1ac73c90d: Bootstrap starting.
I20260925 07:26:28.843077 16055 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 51e2568956264fd9944bb4f1ac73c90d: Neither blocks nor log segments found. Creating new log.
I20260925 07:26:28.843344 16055 log.cc:824] T 00000000000000000000000000000000 P 51e2568956264fd9944bb4f1ac73c90d: Log is configured to *not* fsync() on all Append() calls
I20260925 07:26:28.843767 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 15987
I20260925 07:26:28.843886 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/master-0/wal/instance
I20260925 07:26:28.844059 16055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51e2568956264fd9944bb4f1ac73c90d: No bootstrap required, opened a new log
I20260925 07:26:28.844066 5753 external_mini_cluster.cc:1428] Setting key fbeb73ba55464c8e87795f15d63d4e96
I20260925 07:26:28.845772 16055 raft_consensus.cc:348] T 00000000000000000000000000000000 P 51e2568956264fd9944bb4f1ac73c90d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51e2568956264fd9944bb4f1ac73c90d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 45137 } }
I20260925 07:26:28.845929 16055 raft_consensus.cc:374] T 00000000000000000000000000000000 P 51e2568956264fd9944bb4f1ac73c90d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:26:28.845973 16055 raft_consensus.cc:729] T 00000000000000000000000000000000 P 51e2568956264fd9944bb4f1ac73c90d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51e2568956264fd9944bb4f1ac73c90d, State: Initialized, Role: FOLLOWER
I20260925 07:26:28.846094 16055 consensus_queue.cc:260] T 00000000000000000000000000000000 P 51e2568956264fd9944bb4f1ac73c90d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51e2568956264fd9944bb4f1ac73c90d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 45137 } }
I20260925 07:26:28.846184 16055 raft_consensus.cc:388] T 00000000000000000000000000000000 P 51e2568956264fd9944bb4f1ac73c90d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 07:26:28.846225 16055 raft_consensus.cc:482] T 00000000000000000000000000000000 P 51e2568956264fd9944bb4f1ac73c90d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 07:26:28.846267 16055 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 51e2568956264fd9944bb4f1ac73c90d [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:26:28.846729 16055 raft_consensus.cc:504] T 00000000000000000000000000000000 P 51e2568956264fd9944bb4f1ac73c90d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51e2568956264fd9944bb4f1ac73c90d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 45137 } }
I20260925 07:26:28.846834 16055 leader_election.cc:302] T 00000000000000000000000000000000 P 51e2568956264fd9944bb4f1ac73c90d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 51e2568956264fd9944bb4f1ac73c90d; no voters:
I20260925 07:26:28.846997 16055 leader_election.cc:288] T 00000000000000000000000000000000 P 51e2568956264fd9944bb4f1ac73c90d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 07:26:28.847012 16060 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 51e2568956264fd9944bb4f1ac73c90d [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:26:28.847131 16060 raft_consensus.cc:686] T 00000000000000000000000000000000 P 51e2568956264fd9944bb4f1ac73c90d [term 1 LEADER]: Becoming Leader. State: Replica: 51e2568956264fd9944bb4f1ac73c90d, State: Running, Role: LEADER
I20260925 07:26:28.847205 16060 consensus_queue.cc:237] T 00000000000000000000000000000000 P 51e2568956264fd9944bb4f1ac73c90d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51e2568956264fd9944bb4f1ac73c90d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 45137 } }
I20260925 07:26:28.847255 16055 sys_catalog.cc:564] T 00000000000000000000000000000000 P 51e2568956264fd9944bb4f1ac73c90d [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:26:28.847527 16060 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51e2568956264fd9944bb4f1ac73c90d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "51e2568956264fd9944bb4f1ac73c90d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51e2568956264fd9944bb4f1ac73c90d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 45137 } } }
I20260925 07:26:28.847690 16060 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51e2568956264fd9944bb4f1ac73c90d [sys.catalog]: This master's current role is: LEADER
I20260925 07:26:28.847898 16060 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51e2568956264fd9944bb4f1ac73c90d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 51e2568956264fd9944bb4f1ac73c90d. Latest consensus state: current_term: 1 leader_uuid: "51e2568956264fd9944bb4f1ac73c90d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51e2568956264fd9944bb4f1ac73c90d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 45137 } } }
I20260925 07:26:28.847972 16060 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51e2568956264fd9944bb4f1ac73c90d [sys.catalog]: This master's current role is: LEADER
I20260925 07:26:28.848244 16067 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 07:26:28.848795 16067 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 07:26:28.850545 16067 catalog_manager.cc:1347] Generated new cluster ID: ce3a30de82644a18bdc79a54fa5a5b7f
I20260925 07:26:28.850601 16067 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 07:26:28 dist-test-slave-1drq krb5kdc[15974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321188, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:26:28.854676 16059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:28.844517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60066 (local address 127.5.158.126:45137)
0925 07:26:28.844693 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:28.844700 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:28.844727 (+ 27us) server_negotiation.cc:407] Connection header received
0925 07:26:28.845203 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:28.845209 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:28.845248 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:28.845369 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:28.845910 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:28.847147 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:28.848241 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:28.848861 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:28.848950 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:28.851591 (+ 2641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:26:28.851595 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:26:28.851595 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:26:28.851604 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:26:28.853175 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:26:28.853529 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:26:28.853531 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:26:28.853531 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:26:28.853614 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:26:28.853820 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:26:28.853821 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:26:28.853822 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:26:28.854063 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:26:28.854195 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0925 07:26:28.854310 (+ 115us) server_negotiation.cc:299] Negotiation successful
0925 07:26:28.854422 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":83,"threads_started":1}
I20260925 07:26:28.862867 16067 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 07:26:28.863453 16067 catalog_manager.cc:1504] Loading token signing keys...
I20260925 07:26:28.868314 16067 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 51e2568956264fd9944bb4f1ac73c90d: Generated new TSK 0
I20260925 07:26:28.868465 16067 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.158.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab.
Entry for principal kudu/127.5.158.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab.
I20260925 07:26:28.891170 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.65:0
--local_ip_for_outbound_sockets=127.5.158.65
--webserver_interface=127.5.158.65
--webserver_port=0
--tserver_master_addrs=127.5.158.126:45137
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab
--principal=kudu/127.5.158.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:35759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:26:28.970885 16081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:26:28.971060 16081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:26:28.971081 16081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:26:28.972201 16081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:26:28.972241 16081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:26:28.972267 16081 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.65
I20260925 07:26:28.973691 16081 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:35759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab
--principal=kudu/127.5.158.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.158.65
--webserver_port=0
--tserver_master_addrs=127.5.158.126:45137
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16081
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.65
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:26:28.973924 16081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:26:28.974148 16081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:26:28.978957 16087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:26:28.978967 16089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:26:28.978967 16086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:26:28.979216 16081 server_base.cc:1034] running on GCE node
I20260925 07:26:28.979432 16081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:26:28.979697 16081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:26:28.980872 16081 hybrid_clock.cc:648] HybridClock initialized: now 1790321188980865 us; error 32 us; skew 500 ppm
Sep 25 07:26:28 dist-test-slave-1drq krb5kdc[15974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321188, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:26:28.983237 16081 init.cc:377] Logged in from keytab as kudu/127.5.158.65@KRBTEST.COM (short username kudu)
I20260925 07:26:28.983714 16081 webserver.cc:460] Webserver started at http://127.5.158.65:34557/ using document root <none> and password file <none>
I20260925 07:26:28.983935 16081 fs_manager.cc:359] Metadata directory not provided
I20260925 07:26:28.984001 16081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:26:28.984163 16081 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:26:28.985199 16081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/ts-0/data/instance:
uuid: "67f3930d8c81496bad09ce3ed103df0a"
format_stamp: "Formatted at 2026-09-25 07:26:28 on dist-test-slave-1drq"
server_key: "a5c06900a9d4aad776df26fe13262cc5"
server_key_iv: "c3dd4a2b0d676463d4e57bf6d017bd0c"
server_key_version: "encryptionkey@0"
I20260925 07:26:28.985575 16081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/ts-0/wal/instance:
uuid: "67f3930d8c81496bad09ce3ed103df0a"
format_stamp: "Formatted at 2026-09-25 07:26:28 on dist-test-slave-1drq"
server_key: "a5c06900a9d4aad776df26fe13262cc5"
server_key_iv: "c3dd4a2b0d676463d4e57bf6d017bd0c"
server_key_version: "encryptionkey@0"
I20260925 07:26:28.986763 16081 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 07:26:28.987437 16096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:28.987601 16081 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 07:26:28.987696 16081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/ts-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/ts-0/wal
uuid: "67f3930d8c81496bad09ce3ed103df0a"
format_stamp: "Formatted at 2026-09-25 07:26:28 on dist-test-slave-1drq"
server_key: "a5c06900a9d4aad776df26fe13262cc5"
server_key_iv: "c3dd4a2b0d676463d4e57bf6d017bd0c"
server_key_version: "encryptionkey@0"
I20260925 07:26:28.987778 16081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:26:29.006402 16081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:26:29.008024 16081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:26:29.008178 16081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:26:29.008384 16081 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:26:29.008687 16081 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:26:29.008736 16081 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:29.008774 16081 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:26:29.008803 16081 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:29.014364 16081 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.65:38909
I20260925 07:26:29.014406 16209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.65:38909 every 8 connection(s)
I20260925 07:26:29.014680 16081 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
I20260925 07:26:29.015609 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 16081
I20260925 07:26:29.015714 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/ts-0/wal/instance
I20260925 07:26:29.015903 5753 external_mini_cluster.cc:1428] Setting key 8fea432a83fe80fd5cf50cd4390c06ef
Sep 25 07:26:29 dist-test-slave-1drq krb5kdc[15974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321188, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.65@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.158.66@KRBTEST.COM; defaulting to no policy
I20260925 07:26:29.024631 16059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:29.015266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:55593 (local address 127.5.158.126:45137)
0925 07:26:29.015321 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:29.015323 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:29.015548 (+ 225us) server_negotiation.cc:407] Connection header received
0925 07:26:29.016046 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:29.016047 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:29.016060 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:29.016140 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:29.017055 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:29.017878 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:29.019279 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:29.019626 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:29.019699 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:29.021801 (+ 2102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:26:29.021802 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:26:29.021803 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:26:29.021811 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:26:29.023246 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:26:29.023695 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:26:29.023696 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:26:29.023696 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:26:29.023762 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:26:29.024021 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:26:29.024022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:26:29.024023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:26:29.024131 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:26:29.024206 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 07:26:29.024459 (+ 253us) server_negotiation.cc:299] Negotiation successful
0925 07:26:29.024539 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260925 07:26:29.024922 16212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:29.015265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:45137 (local address 127.5.158.65:55593)
0925 07:26:29.015415 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:29.015459 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:29.015941 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:29.016193 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:29.016198 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:29.016791 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:29.016957 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:29.016961 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:29.017979 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:29.017980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:29.019193 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:29.019196 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:29.019706 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:29.019707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:29.019798 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:29.020294 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:29.020302 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:29.021695 (+ 1393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:26:29.023366 (+ 1671us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:26:29.023373 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:26:29.023375 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:26:29.023601 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:26:29.023832 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:26:29.023833 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:26:29.023833 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:26:29.023954 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:26:29.024197 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:26:29.024203 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:26:29.024335 (+ 132us) client_negotiation.cc:769] Sending connection context
0925 07:26:29.024439 (+ 104us) client_negotiation.cc:240] Negotiation successful
0925 07:26:29.024582 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":66,"threads_started":1}
I20260925 07:26:29.025398 16210 heartbeater.cc:347] Connected to a master server at 127.5.158.126:45137
I20260925 07:26:29.025468 16210 heartbeater.cc:464] Registering TS with master...
I20260925 07:26:29.025645 16210 heartbeater.cc:511] Master 127.5.158.126:45137 requested a full tablet report, sending...
I20260925 07:26:29.026139 16019 ts_manager.cc:194] Registered new tserver with Master: 67f3930d8c81496bad09ce3ed103df0a (127.5.158.65:38909)
I20260925 07:26:29.026741 16019 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.65@KRBTEST.COM'} at 127.5.158.65:55593
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab.
Entry for principal kudu/127.5.158.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab.
I20260925 07:26:29.040323 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.66:0
--local_ip_for_outbound_sockets=127.5.158.66
--webserver_interface=127.5.158.66
--webserver_port=0
--tserver_master_addrs=127.5.158.126:45137
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab
--principal=kudu/127.5.158.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:35759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:26:29.114398 16215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:26:29.114564 16215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:26:29.114585 16215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:26:29.115581 16215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:26:29.115615 16215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:26:29.115659 16215 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.66
I20260925 07:26:29.116909 16215 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:35759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab
--principal=kudu/127.5.158.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.158.66
--webserver_port=0
--tserver_master_addrs=127.5.158.126:45137
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16215
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.66
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:26:29.117069 16215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:26:29.117302 16215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:26:29.121850 16221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:26:29.121891 16215 server_base.cc:1034] running on GCE node
W20260925 07:26:29.121857 16220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:26:29.121898 16223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:26:29.122136 16215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:26:29.122359 16215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:26:29.123485 16215 hybrid_clock.cc:648] HybridClock initialized: now 1790321189123456 us; error 44 us; skew 500 ppm
Sep 25 07:26:29 dist-test-slave-1drq krb5kdc[15974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321189, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:26:29.125700 16215 init.cc:377] Logged in from keytab as kudu/127.5.158.66@KRBTEST.COM (short username kudu)
I20260925 07:26:29.126149 16215 webserver.cc:460] Webserver started at http://127.5.158.66:33801/ using document root <none> and password file <none>
I20260925 07:26:29.126339 16215 fs_manager.cc:359] Metadata directory not provided
I20260925 07:26:29.126401 16215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:26:29.126536 16215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:26:29.127449 16215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/ts-1/data/instance:
uuid: "0190068188474cceba71c9e9913742b8"
format_stamp: "Formatted at 2026-09-25 07:26:29 on dist-test-slave-1drq"
server_key: "081bfb0bfb0143515b962f1789324e09"
server_key_iv: "9e0399939c4a9b015160bf177483ae2f"
server_key_version: "encryptionkey@0"
I20260925 07:26:29.127803 16215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/ts-1/wal/instance:
uuid: "0190068188474cceba71c9e9913742b8"
format_stamp: "Formatted at 2026-09-25 07:26:29 on dist-test-slave-1drq"
server_key: "081bfb0bfb0143515b962f1789324e09"
server_key_iv: "9e0399939c4a9b015160bf177483ae2f"
server_key_version: "encryptionkey@0"
I20260925 07:26:29.129046 16215 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 07:26:29.129808 16230 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:29.129976 16215 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 07:26:29.130061 16215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/ts-1/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/ts-1/wal
uuid: "0190068188474cceba71c9e9913742b8"
format_stamp: "Formatted at 2026-09-25 07:26:29 on dist-test-slave-1drq"
server_key: "081bfb0bfb0143515b962f1789324e09"
server_key_iv: "9e0399939c4a9b015160bf177483ae2f"
server_key_version: "encryptionkey@0"
I20260925 07:26:29.130136 16215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:26:29.158681 16215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:26:29.160386 16215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:26:29.160547 16215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:26:29.160789 16215 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:26:29.161115 16215 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:26:29.161173 16215 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:29.161216 16215 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:26:29.161245 16215 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:29.166814 16215 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.66:43375
I20260925 07:26:29.166859 16343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.66:43375 every 8 connection(s)
I20260925 07:26:29.167135 16215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
Sep 25 07:26:29 dist-test-slave-1drq krb5kdc[15974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321189, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.66@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:26:29.174821 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 16215
I20260925 07:26:29.174979 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/ts-1/wal/instance
I20260925 07:26:29.175256 5753 external_mini_cluster.cc:1428] Setting key 2231d121d12b697b71bc053da3186423
I20260925 07:26:29.176205 16059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:29.167761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:48541 (local address 127.5.158.126:45137)
0925 07:26:29.167842 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:29.167845 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:29.168042 (+ 197us) server_negotiation.cc:407] Connection header received
0925 07:26:29.168686 (+ 644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:29.168688 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:29.168699 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:29.168756 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:29.169763 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:29.170256 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:29.171303 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:29.171509 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:29.171593 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:29.173542 (+ 1949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:26:29.173544 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:26:29.173544 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:26:29.173549 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:26:29.174583 (+ 1034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:26:29.175107 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:26:29.175108 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:26:29.175108 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:26:29.175202 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:26:29.175473 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:26:29.175475 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:26:29.175475 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:26:29.175613 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:26:29.175674 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 07:26:29.175995 (+ 321us) server_negotiation.cc:299] Negotiation successful
0925 07:26:29.176084 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 07:26:29.176232 16346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:29.167750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:45137 (local address 127.5.158.66:48541)
0925 07:26:29.167914 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:29.167970 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:29.168598 (+ 628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:29.168829 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:29.168833 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:29.169484 (+ 651us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:29.169662 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:29.169666 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:29.170368 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:29.170369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:29.171201 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:29.171203 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:29.171603 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:29.171604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:29.171726 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:29.172147 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:29.172154 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:29.173449 (+ 1295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:26:29.174668 (+ 1219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:26:29.174675 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:26:29.174677 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:26:29.175020 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:26:29.175268 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:26:29.175269 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:26:29.175269 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:26:29.175401 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:26:29.175671 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:26:29.175677 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:26:29.175823 (+ 146us) client_negotiation.cc:769] Sending connection context
0925 07:26:29.175947 (+ 124us) client_negotiation.cc:240] Negotiation successful
0925 07:26:29.176086 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":82,"threads_started":1}
I20260925 07:26:29.176997 16344 heartbeater.cc:347] Connected to a master server at 127.5.158.126:45137
I20260925 07:26:29.177109 16344 heartbeater.cc:464] Registering TS with master...
I20260925 07:26:29.177285 16344 heartbeater.cc:511] Master 127.5.158.126:45137 requested a full tablet report, sending...
I20260925 07:26:29.177733 16019 ts_manager.cc:194] Registered new tserver with Master: 0190068188474cceba71c9e9913742b8 (127.5.158.66:43375)
I20260925 07:26:29.178232 16019 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.66@KRBTEST.COM'} at 127.5.158.66:48541
WARNING: no policy specified for kudu/127.5.158.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab.
Entry for principal kudu/127.5.158.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab.
I20260925 07:26:29.197824 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.67:0
--local_ip_for_outbound_sockets=127.5.158.67
--webserver_interface=127.5.158.67
--webserver_port=0
--tserver_master_addrs=127.5.158.126:45137
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab
--principal=kudu/127.5.158.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:35759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:26:29.271499 16349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:26:29.271668 16349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:26:29.271688 16349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:26:29.272688 16349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:26:29.272720 16349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:26:29.272742 16349 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.67
I20260925 07:26:29.274055 16349 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:35759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab
--principal=kudu/127.5.158.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.158.67
--webserver_port=0
--tserver_master_addrs=127.5.158.126:45137
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16349
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.67
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:26:29.274245 16349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:26:29.274425 16349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:26:29.279040 16357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:26:29.279101 16355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:26:29.279101 16354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:26:29.279402 16349 server_base.cc:1034] running on GCE node
I20260925 07:26:29.279593 16349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:26:29.279801 16349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:26:29.280927 16349 hybrid_clock.cc:648] HybridClock initialized: now 1790321189280916 us; error 28 us; skew 500 ppm
Sep 25 07:26:29 dist-test-slave-1drq krb5kdc[15974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321189, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:26:29.283228 16349 init.cc:377] Logged in from keytab as kudu/127.5.158.67@KRBTEST.COM (short username kudu)
I20260925 07:26:29.283620 16349 webserver.cc:460] Webserver started at http://127.5.158.67:43607/ using document root <none> and password file <none>
I20260925 07:26:29.283888 16349 fs_manager.cc:359] Metadata directory not provided
I20260925 07:26:29.283962 16349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:26:29.284101 16349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:26:29.285025 16349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/ts-2/data/instance:
uuid: "380ad4048218433798d886a0672f4ea8"
format_stamp: "Formatted at 2026-09-25 07:26:29 on dist-test-slave-1drq"
server_key: "750610d6c85f0ac135e4e9d50aea4552"
server_key_iv: "d388e8d42c61a19a6a74186de9a48b53"
server_key_version: "encryptionkey@0"
I20260925 07:26:29.285431 16349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/ts-2/wal/instance:
uuid: "380ad4048218433798d886a0672f4ea8"
format_stamp: "Formatted at 2026-09-25 07:26:29 on dist-test-slave-1drq"
server_key: "750610d6c85f0ac135e4e9d50aea4552"
server_key_iv: "d388e8d42c61a19a6a74186de9a48b53"
server_key_version: "encryptionkey@0"
I20260925 07:26:29.286739 16349 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 07:26:29.287532 16364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:29.287758 16349 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 07:26:29.287856 16349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/ts-2/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/ts-2/wal
uuid: "380ad4048218433798d886a0672f4ea8"
format_stamp: "Formatted at 2026-09-25 07:26:29 on dist-test-slave-1drq"
server_key: "750610d6c85f0ac135e4e9d50aea4552"
server_key_iv: "d388e8d42c61a19a6a74186de9a48b53"
server_key_version: "encryptionkey@0"
I20260925 07:26:29.287938 16349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:26:29.303745 16349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:26:29.305305 16349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:26:29.305452 16349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:26:29.305639 16349 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:26:29.305894 16349 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:26:29.305943 16349 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:29.305974 16349 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:26:29.305997 16349 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:26:29.311599 16349 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.67:46263
I20260925 07:26:29.311630 16477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.67:46263 every 8 connection(s)
I20260925 07:26:29.311919 16349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
Sep 25 07:26:29 dist-test-slave-1drq krb5kdc[15974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321189, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.67@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:26:29.320430 16059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:29.312555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:39355 (local address 127.5.158.126:45137)
0925 07:26:29.312639 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:29.312641 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:29.312849 (+ 208us) server_negotiation.cc:407] Connection header received
0925 07:26:29.313480 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:29.313481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:29.313493 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:29.313559 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:29.314437 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:29.314927 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:29.315868 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:29.316073 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:29.316158 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:29.318035 (+ 1877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:26:29.318036 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:26:29.318037 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:26:29.318044 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:26:29.319164 (+ 1120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:26:29.319535 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:26:29.319536 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:26:29.319536 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:26:29.319591 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:26:29.319838 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:26:29.319839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:26:29.319839 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:26:29.319949 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:26:29.320024 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 07:26:29.320272 (+ 248us) server_negotiation.cc:299] Negotiation successful
0925 07:26:29.320345 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260925 07:26:29.320612 16480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:29.312518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:45137 (local address 127.5.158.67:39355)
0925 07:26:29.312720 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:29.312761 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:29.313382 (+ 621us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:29.313634 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:29.313643 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:29.314172 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:26:29.314337 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:29.314341 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:29.315024 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:29.315025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:29.315781 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:29.315783 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:29.316165 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:29.316166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:29.316289 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:29.316672 (+ 383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:26:29.316679 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:26:29.317926 (+ 1247us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:26:29.319250 (+ 1324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:26:29.319255 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:26:29.319257 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:26:29.319437 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:26:29.319654 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:26:29.319654 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:26:29.319655 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:26:29.319771 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:26:29.320015 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:26:29.320021 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:26:29.320150 (+ 129us) client_negotiation.cc:769] Sending connection context
0925 07:26:29.320253 (+ 103us) client_negotiation.cc:240] Negotiation successful
0925 07:26:29.320402 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":83,"threads_started":1}
I20260925 07:26:29.321141 16478 heartbeater.cc:347] Connected to a master server at 127.5.158.126:45137
I20260925 07:26:29.321237 16478 heartbeater.cc:464] Registering TS with master...
I20260925 07:26:29.321395 16478 heartbeater.cc:511] Master 127.5.158.126:45137 requested a full tablet report, sending...
I20260925 07:26:29.321727 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 16349
I20260925 07:26:29.321763 16019 ts_manager.cc:194] Registered new tserver with Master: 380ad4048218433798d886a0672f4ea8 (127.5.158.67:46263)
I20260925 07:26:29.321838 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/ts-2/wal/instance
I20260925 07:26:29.322012 5753 external_mini_cluster.cc:1428] Setting key 5f2c3afce27520eb1fcec3ff20c06f78
I20260925 07:26:29.322163 16019 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.67@KRBTEST.COM'} at 127.5.158.67:39355
I20260925 07:26:29.322862 5753 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 07:26:29.329043 16059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:29.323965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60076 (local address 127.5.158.126:45137)
0925 07:26:29.324021 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:29.324024 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:29.324091 (+ 67us) server_negotiation.cc:407] Connection header received
0925 07:26:29.324121 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:29.324122 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:29.324130 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:29.324170 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:26:29.324644 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:29.325011 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:29.325712 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:29.325864 (+ 152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:29.325912 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:29.326877 (+ 965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:26:29.326878 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:26:29.326878 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:26:29.326883 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:26:29.328195 (+ 1312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:26:29.328475 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:26:29.328476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:26:29.328476 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:26:29.328532 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:26:29.328671 (+ 139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:26:29.328672 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:26:29.328672 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:26:29.328772 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:26:29.328841 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 07:26:29.328923 (+ 82us) server_negotiation.cc:299] Negotiation successful
0925 07:26:29.328971 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260925 07:26:29.330232 16019 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60076:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 07:26:29.331733 16019 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 07:26:29.337661 16486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:29.333831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:38909 (local address 127.0.0.1:48054)
0925 07:26:29.333974 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:29.334006 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:29.334062 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:29.334257 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:29.334261 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:29.334263 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:26:29.334315 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:29.334317 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:29.335772 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:29.335773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:29.336434 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:29.336436 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:29.337356 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:29.337357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:29.337438 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:29.337449 (+ 11us) client_negotiation.cc:769] Sending connection context
0925 07:26:29.337519 (+ 70us) client_negotiation.cc:240] Negotiation successful
0925 07:26:29.337567 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":55,"threads_started":1}
I20260925 07:26:29.337694 16489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:29.333958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48054 (local address 127.5.158.65:38909)
0925 07:26:29.334091 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:29.334096 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:29.334106 (+ 10us) server_negotiation.cc:407] Connection header received
0925 07:26:29.334146 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:29.334147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:29.334162 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:29.334239 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:26:29.334379 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:29.335679 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:29.336529 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:29.337256 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:29.337326 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:29.337396 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 07:26:29.337548 (+ 152us) server_negotiation.cc:299] Negotiation successful
0925 07:26:29.337590 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":72,"threads_started":1}
I20260925 07:26:29.337841 16488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:29.333904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59488 (local address 127.5.158.67:46263)
0925 07:26:29.334267 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:29.334271 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:29.334278 (+ 7us) server_negotiation.cc:407] Connection header received
0925 07:26:29.334307 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:29.334308 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:29.334320 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:29.334402 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:26:29.334588 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:29.335637 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:29.336764 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:29.337368 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:29.337434 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:29.337486 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 07:26:29.337710 (+ 224us) server_negotiation.cc:299] Negotiation successful
0925 07:26:29.337757 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":70,"threads_started":1}
I20260925 07:26:29.337903 16487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:29.333831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.67:46263 (local address 127.0.0.1:59488)
0925 07:26:29.334112 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:29.334122 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:29.334169 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:29.334460 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:29.334460 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:29.334462 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:26:29.334497 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:29.334498 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:29.335730 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:29.335731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:29.336673 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:29.336675 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:29.337519 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:29.337520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:29.337574 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:29.337577 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 07:26:29.337622 (+ 45us) client_negotiation.cc:240] Negotiation successful
0925 07:26:29.337668 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"mutex_wait_us":2,"thread_start_us":64,"threads_started":1}
I20260925 07:26:29.338366 16144 tablet_service.cc:1467] Processing CreateTablet for tablet 9df2e12a84f94dcc8817974a9dd0b122 (DEFAULT_TABLE table=test-table [id=020006659ed1445a866d9e78b7005640]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:26:29.338433 16412 tablet_service.cc:1467] Processing CreateTablet for tablet 9df2e12a84f94dcc8817974a9dd0b122 (DEFAULT_TABLE table=test-table [id=020006659ed1445a866d9e78b7005640]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:26:29.338675 16490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:29.334004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.66:43375 (local address 127.0.0.1:50714)
0925 07:26:29.334364 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:29.334372 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:29.334408 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:29.334631 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:29.334632 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:29.334633 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:26:29.334673 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:29.334674 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:29.335507 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:29.335508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:29.336772 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:29.336773 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:29.338442 (+ 1669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:29.338443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:29.338504 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:29.338508 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 07:26:29.338576 (+ 68us) client_negotiation.cc:240] Negotiation successful
0925 07:26:29.338614 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":56,"threads_started":1}
I20260925 07:26:29.338742 16144 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9df2e12a84f94dcc8817974a9dd0b122. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:26:29.338768 16412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9df2e12a84f94dcc8817974a9dd0b122. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:26:29.338987 16491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:29.334179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50714 (local address 127.5.158.66:43375)
0925 07:26:29.334496 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:29.334499 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:29.334506 (+ 7us) server_negotiation.cc:407] Connection header received
0925 07:26:29.334553 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:29.334555 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:29.334569 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:29.334633 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:26:29.334734 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:29.335401 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:29.336833 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:29.337663 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:29.338416 (+ 753us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:29.338481 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 07:26:29.338868 (+ 387us) server_negotiation.cc:299] Negotiation successful
0925 07:26:29.338918 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":90,"threads_started":1}
I20260925 07:26:29.341113 16493 tablet_bootstrap.cc:492] T 9df2e12a84f94dcc8817974a9dd0b122 P 380ad4048218433798d886a0672f4ea8: Bootstrap starting.
I20260925 07:26:29.341548 16278 tablet_service.cc:1467] Processing CreateTablet for tablet 9df2e12a84f94dcc8817974a9dd0b122 (DEFAULT_TABLE table=test-table [id=020006659ed1445a866d9e78b7005640]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:26:29.341665 16493 tablet_bootstrap.cc:654] T 9df2e12a84f94dcc8817974a9dd0b122 P 380ad4048218433798d886a0672f4ea8: Neither blocks nor log segments found. Creating new log.
I20260925 07:26:29.341854 16278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9df2e12a84f94dcc8817974a9dd0b122. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:26:29.342173 16493 log.cc:824] T 9df2e12a84f94dcc8817974a9dd0b122 P 380ad4048218433798d886a0672f4ea8: Log is configured to *not* fsync() on all Append() calls
I20260925 07:26:29.342564 16492 tablet_bootstrap.cc:492] T 9df2e12a84f94dcc8817974a9dd0b122 P 67f3930d8c81496bad09ce3ed103df0a: Bootstrap starting.
I20260925 07:26:29.343053 16492 tablet_bootstrap.cc:654] T 9df2e12a84f94dcc8817974a9dd0b122 P 67f3930d8c81496bad09ce3ed103df0a: Neither blocks nor log segments found. Creating new log.
I20260925 07:26:29.343472 16492 log.cc:824] T 9df2e12a84f94dcc8817974a9dd0b122 P 67f3930d8c81496bad09ce3ed103df0a: Log is configured to *not* fsync() on all Append() calls
I20260925 07:26:29.344019 16493 tablet_bootstrap.cc:492] T 9df2e12a84f94dcc8817974a9dd0b122 P 380ad4048218433798d886a0672f4ea8: No bootstrap required, opened a new log
I20260925 07:26:29.344098 16493 ts_tablet_manager.cc:1397] T 9df2e12a84f94dcc8817974a9dd0b122 P 380ad4048218433798d886a0672f4ea8: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260925 07:26:29.344117 16492 tablet_bootstrap.cc:492] T 9df2e12a84f94dcc8817974a9dd0b122 P 67f3930d8c81496bad09ce3ed103df0a: No bootstrap required, opened a new log
I20260925 07:26:29.344172 16492 ts_tablet_manager.cc:1397] T 9df2e12a84f94dcc8817974a9dd0b122 P 67f3930d8c81496bad09ce3ed103df0a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 07:26:29.344493 16496 tablet_bootstrap.cc:492] T 9df2e12a84f94dcc8817974a9dd0b122 P 0190068188474cceba71c9e9913742b8: Bootstrap starting.
I20260925 07:26:29.345058 16496 tablet_bootstrap.cc:654] T 9df2e12a84f94dcc8817974a9dd0b122 P 0190068188474cceba71c9e9913742b8: Neither blocks nor log segments found. Creating new log.
I20260925 07:26:29.345727 16496 log.cc:824] T 9df2e12a84f94dcc8817974a9dd0b122 P 0190068188474cceba71c9e9913742b8: Log is configured to *not* fsync() on all Append() calls
I20260925 07:26:29.345952 16492 raft_consensus.cc:348] T 9df2e12a84f94dcc8817974a9dd0b122 P 67f3930d8c81496bad09ce3ed103df0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67f3930d8c81496bad09ce3ed103df0a" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 38909 } } peers { permanent_uuid: "380ad4048218433798d886a0672f4ea8" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 46263 } } peers { permanent_uuid: "0190068188474cceba71c9e9913742b8" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 43375 } }
I20260925 07:26:29.346131 16492 raft_consensus.cc:374] T 9df2e12a84f94dcc8817974a9dd0b122 P 67f3930d8c81496bad09ce3ed103df0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:26:29.346037 16493 raft_consensus.cc:348] T 9df2e12a84f94dcc8817974a9dd0b122 P 380ad4048218433798d886a0672f4ea8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67f3930d8c81496bad09ce3ed103df0a" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 38909 } } peers { permanent_uuid: "380ad4048218433798d886a0672f4ea8" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 46263 } } peers { permanent_uuid: "0190068188474cceba71c9e9913742b8" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 43375 } }
I20260925 07:26:29.346171 16492 raft_consensus.cc:729] T 9df2e12a84f94dcc8817974a9dd0b122 P 67f3930d8c81496bad09ce3ed103df0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67f3930d8c81496bad09ce3ed103df0a, State: Initialized, Role: FOLLOWER
I20260925 07:26:29.346199 16493 raft_consensus.cc:374] T 9df2e12a84f94dcc8817974a9dd0b122 P 380ad4048218433798d886a0672f4ea8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:26:29.346239 16493 raft_consensus.cc:729] T 9df2e12a84f94dcc8817974a9dd0b122 P 380ad4048218433798d886a0672f4ea8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 380ad4048218433798d886a0672f4ea8, State: Initialized, Role: FOLLOWER
I20260925 07:26:29.346271 16492 consensus_queue.cc:260] T 9df2e12a84f94dcc8817974a9dd0b122 P 67f3930d8c81496bad09ce3ed103df0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67f3930d8c81496bad09ce3ed103df0a" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 38909 } } peers { permanent_uuid: "380ad4048218433798d886a0672f4ea8" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 46263 } } peers { permanent_uuid: "0190068188474cceba71c9e9913742b8" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 43375 } }
I20260925 07:26:29.346356 16493 consensus_queue.cc:260] T 9df2e12a84f94dcc8817974a9dd0b122 P 380ad4048218433798d886a0672f4ea8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67f3930d8c81496bad09ce3ed103df0a" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 38909 } } peers { permanent_uuid: "380ad4048218433798d886a0672f4ea8" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 46263 } } peers { permanent_uuid: "0190068188474cceba71c9e9913742b8" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 43375 } }
I20260925 07:26:29.346539 16492 ts_tablet_manager.cc:1428] T 9df2e12a84f94dcc8817974a9dd0b122 P 67f3930d8c81496bad09ce3ed103df0a: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 07:26:29.346556 16496 tablet_bootstrap.cc:492] T 9df2e12a84f94dcc8817974a9dd0b122 P 0190068188474cceba71c9e9913742b8: No bootstrap required, opened a new log
I20260925 07:26:29.346568 16210 heartbeater.cc:503] Master 127.5.158.126:45137 was elected leader, sending a full tablet report...
I20260925 07:26:29.346623 16496 ts_tablet_manager.cc:1397] T 9df2e12a84f94dcc8817974a9dd0b122 P 0190068188474cceba71c9e9913742b8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 07:26:29.346628 16493 ts_tablet_manager.cc:1428] T 9df2e12a84f94dcc8817974a9dd0b122 P 380ad4048218433798d886a0672f4ea8: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 07:26:29.346746 16478 heartbeater.cc:503] Master 127.5.158.126:45137 was elected leader, sending a full tablet report...
I20260925 07:26:29.348474 16496 raft_consensus.cc:348] T 9df2e12a84f94dcc8817974a9dd0b122 P 0190068188474cceba71c9e9913742b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67f3930d8c81496bad09ce3ed103df0a" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 38909 } } peers { permanent_uuid: "380ad4048218433798d886a0672f4ea8" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 46263 } } peers { permanent_uuid: "0190068188474cceba71c9e9913742b8" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 43375 } }
I20260925 07:26:29.348601 16496 raft_consensus.cc:374] T 9df2e12a84f94dcc8817974a9dd0b122 P 0190068188474cceba71c9e9913742b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:26:29.348654 16496 raft_consensus.cc:729] T 9df2e12a84f94dcc8817974a9dd0b122 P 0190068188474cceba71c9e9913742b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0190068188474cceba71c9e9913742b8, State: Initialized, Role: FOLLOWER
I20260925 07:26:29.348757 16496 consensus_queue.cc:260] T 9df2e12a84f94dcc8817974a9dd0b122 P 0190068188474cceba71c9e9913742b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67f3930d8c81496bad09ce3ed103df0a" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 38909 } } peers { permanent_uuid: "380ad4048218433798d886a0672f4ea8" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 46263 } } peers { permanent_uuid: "0190068188474cceba71c9e9913742b8" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 43375 } }
I20260925 07:26:29.349018 16496 ts_tablet_manager.cc:1428] T 9df2e12a84f94dcc8817974a9dd0b122 P 0190068188474cceba71c9e9913742b8: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 07:26:29.349044 16344 heartbeater.cc:503] Master 127.5.158.126:45137 was elected leader, sending a full tablet report...
W20260925 07:26:29.417743 16345 tablet.cc:2369] T 9df2e12a84f94dcc8817974a9dd0b122 P 0190068188474cceba71c9e9913742b8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 07:26:29.418881 16499 raft_consensus.cc:482] T 9df2e12a84f94dcc8817974a9dd0b122 P 380ad4048218433798d886a0672f4ea8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 07:26:29.418992 16499 raft_consensus.cc:504] T 9df2e12a84f94dcc8817974a9dd0b122 P 380ad4048218433798d886a0672f4ea8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67f3930d8c81496bad09ce3ed103df0a" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 38909 } } peers { permanent_uuid: "380ad4048218433798d886a0672f4ea8" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 46263 } } peers { permanent_uuid: "0190068188474cceba71c9e9913742b8" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 43375 } }
I20260925 07:26:29.419343 16499 leader_election.cc:288] T 9df2e12a84f94dcc8817974a9dd0b122 P 380ad4048218433798d886a0672f4ea8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 67f3930d8c81496bad09ce3ed103df0a (127.5.158.65:38909), 0190068188474cceba71c9e9913742b8 (127.5.158.66:43375)
I20260925 07:26:29.423137 16489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:29.419725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:50583 (local address 127.5.158.65:38909)
0925 07:26:29.419772 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:29.419774 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:29.419782 (+ 8us) server_negotiation.cc:407] Connection header received
0925 07:26:29.419808 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:29.419809 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:29.419825 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:29.419881 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:26:29.420052 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:29.420708 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:29.422267 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:29.422796 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:29.422889 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:29.422896 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 07:26:29.423044 (+ 148us) server_negotiation.cc:299] Negotiation successful
0925 07:26:29.423079 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 07:26:29.423137 16491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:29.419714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:53365 (local address 127.5.158.66:43375)
0925 07:26:29.419763 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:29.419765 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:29.419791 (+ 26us) server_negotiation.cc:407] Connection header received
0925 07:26:29.419854 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:29.419855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:29.419863 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:29.419908 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:26:29.420052 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:29.420709 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:29.422267 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:29.422823 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:29.422888 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:29.422896 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 07:26:29.423044 (+ 148us) server_negotiation.cc:299] Negotiation successful
0925 07:26:29.423078 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 07:26:29.423154 16480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:29.419511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:38909 (local address 127.5.158.67:50583)
0925 07:26:29.419560 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:29.419572 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:29.419633 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:29.419885 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:29.419885 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:29.419901 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:26:29.419970 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:29.419972 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:29.420793 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:29.420794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:29.422163 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:29.422167 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:29.422901 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:29.422901 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:29.422971 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:29.422973 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 07:26:29.423024 (+ 51us) client_negotiation.cc:240] Negotiation successful
0925 07:26:29.423067 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":122}
I20260925 07:26:29.423164 16501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:29.419603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.66:43375 (local address 127.5.158.67:53365)
0925 07:26:29.419748 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:29.419756 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 07:26:29.419799 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:26:29.419922 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:26:29.419923 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:26:29.419924 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:26:29.419985 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:29.419987 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:29.420793 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:29.420794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:29.422165 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:26:29.422167 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:26:29.422906 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:26:29.422907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:26:29.422969 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:29.422972 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 07:26:29.423024 (+ 52us) client_negotiation.cc:240] Negotiation successful
0925 07:26:29.423066 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":87,"thread_start_us":100,"threads_started":1}
I20260925 07:26:29.423578 16164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9df2e12a84f94dcc8817974a9dd0b122" candidate_uuid: "380ad4048218433798d886a0672f4ea8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67f3930d8c81496bad09ce3ed103df0a" is_pre_election: true
I20260925 07:26:29.423689 16164 raft_consensus.cc:2454] T 9df2e12a84f94dcc8817974a9dd0b122 P 67f3930d8c81496bad09ce3ed103df0a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 380ad4048218433798d886a0672f4ea8 in term 0.
I20260925 07:26:29.423646 16298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9df2e12a84f94dcc8817974a9dd0b122" candidate_uuid: "380ad4048218433798d886a0672f4ea8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0190068188474cceba71c9e9913742b8" is_pre_election: true
I20260925 07:26:29.423753 16298 raft_consensus.cc:2454] T 9df2e12a84f94dcc8817974a9dd0b122 P 0190068188474cceba71c9e9913742b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 380ad4048218433798d886a0672f4ea8 in term 0.
I20260925 07:26:29.423909 16366 leader_election.cc:302] T 9df2e12a84f94dcc8817974a9dd0b122 P 380ad4048218433798d886a0672f4ea8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 380ad4048218433798d886a0672f4ea8, 67f3930d8c81496bad09ce3ed103df0a; no voters:
I20260925 07:26:29.424022 16499 raft_consensus.cc:2781] T 9df2e12a84f94dcc8817974a9dd0b122 P 380ad4048218433798d886a0672f4ea8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 07:26:29.424091 16499 raft_consensus.cc:482] T 9df2e12a84f94dcc8817974a9dd0b122 P 380ad4048218433798d886a0672f4ea8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 07:26:29.424115 16499 raft_consensus.cc:3037] T 9df2e12a84f94dcc8817974a9dd0b122 P 380ad4048218433798d886a0672f4ea8 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:26:29.424746 16499 raft_consensus.cc:504] T 9df2e12a84f94dcc8817974a9dd0b122 P 380ad4048218433798d886a0672f4ea8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67f3930d8c81496bad09ce3ed103df0a" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 38909 } } peers { permanent_uuid: "380ad4048218433798d886a0672f4ea8" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 46263 } } peers { permanent_uuid: "0190068188474cceba71c9e9913742b8" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 43375 } }
I20260925 07:26:29.424876 16499 leader_election.cc:288] T 9df2e12a84f94dcc8817974a9dd0b122 P 380ad4048218433798d886a0672f4ea8 [CANDIDATE]: Term 1 election: Requested vote from peers 67f3930d8c81496bad09ce3ed103df0a (127.5.158.65:38909), 0190068188474cceba71c9e9913742b8 (127.5.158.66:43375)
I20260925 07:26:29.425014 16164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9df2e12a84f94dcc8817974a9dd0b122" candidate_uuid: "380ad4048218433798d886a0672f4ea8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67f3930d8c81496bad09ce3ed103df0a"
I20260925 07:26:29.425122 16164 raft_consensus.cc:3037] T 9df2e12a84f94dcc8817974a9dd0b122 P 67f3930d8c81496bad09ce3ed103df0a [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:26:29.425762 16164 raft_consensus.cc:2454] T 9df2e12a84f94dcc8817974a9dd0b122 P 67f3930d8c81496bad09ce3ed103df0a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 380ad4048218433798d886a0672f4ea8 in term 1.
I20260925 07:26:29.425871 16298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9df2e12a84f94dcc8817974a9dd0b122" candidate_uuid: "380ad4048218433798d886a0672f4ea8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0190068188474cceba71c9e9913742b8"
I20260925 07:26:29.425913 16366 leader_election.cc:302] T 9df2e12a84f94dcc8817974a9dd0b122 P 380ad4048218433798d886a0672f4ea8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 380ad4048218433798d886a0672f4ea8, 67f3930d8c81496bad09ce3ed103df0a; no voters:
I20260925 07:26:29.425954 16298 raft_consensus.cc:3037] T 9df2e12a84f94dcc8817974a9dd0b122 P 0190068188474cceba71c9e9913742b8 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:26:29.426589 16298 raft_consensus.cc:2454] T 9df2e12a84f94dcc8817974a9dd0b122 P 0190068188474cceba71c9e9913742b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 380ad4048218433798d886a0672f4ea8 in term 1.
I20260925 07:26:29.426699 16499 raft_consensus.cc:2781] T 9df2e12a84f94dcc8817974a9dd0b122 P 380ad4048218433798d886a0672f4ea8 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:26:29.426903 16499 raft_consensus.cc:686] T 9df2e12a84f94dcc8817974a9dd0b122 P 380ad4048218433798d886a0672f4ea8 [term 1 LEADER]: Becoming Leader. State: Replica: 380ad4048218433798d886a0672f4ea8, State: Running, Role: LEADER
I20260925 07:26:29.426996 16499 consensus_queue.cc:237] T 9df2e12a84f94dcc8817974a9dd0b122 P 380ad4048218433798d886a0672f4ea8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67f3930d8c81496bad09ce3ed103df0a" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 38909 } } peers { permanent_uuid: "380ad4048218433798d886a0672f4ea8" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 46263 } } peers { permanent_uuid: "0190068188474cceba71c9e9913742b8" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 43375 } }
I20260925 07:26:29.427802 16018 catalog_manager.cc:5495] T 9df2e12a84f94dcc8817974a9dd0b122 P 380ad4048218433798d886a0672f4ea8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 380ad4048218433798d886a0672f4ea8 (127.5.158.67). New cstate: current_term: 1 leader_uuid: "380ad4048218433798d886a0672f4ea8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67f3930d8c81496bad09ce3ed103df0a" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 38909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "380ad4048218433798d886a0672f4ea8" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 46263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0190068188474cceba71c9e9913742b8" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 43375 } health_report { overall_health: UNKNOWN } } }
I20260925 07:26:29.452005 16488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:29.449884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59498 (local address 127.5.158.67:46263)
0925 07:26:29.449925 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 07:26:29.449927 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:26:29.449934 (+ 7us) server_negotiation.cc:407] Connection header received
0925 07:26:29.449967 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:26:29.449968 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:26:29.449981 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:26:29.450065 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 07:26:29.450168 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:29.450588 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:29.451277 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:26:29.451459 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:26:29.451532 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:26:29.451704 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 07:26:29.451789 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 07:26:29.451872 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0925 07:26:29.451894 (+ 22us) server_negotiation.cc:299] Negotiation successful
0925 07:26:29.451911 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 07:26:29.453461 16298 raft_consensus.cc:1261] T 9df2e12a84f94dcc8817974a9dd0b122 P 0190068188474cceba71c9e9913742b8 [term 1 FOLLOWER]: Refusing update from remote peer 380ad4048218433798d886a0672f4ea8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:26:29.453478 16164 raft_consensus.cc:1261] T 9df2e12a84f94dcc8817974a9dd0b122 P 67f3930d8c81496bad09ce3ed103df0a [term 1 FOLLOWER]: Refusing update from remote peer 380ad4048218433798d886a0672f4ea8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:26:29.453788 16499 consensus_queue.cc:1035] T 9df2e12a84f94dcc8817974a9dd0b122 P 380ad4048218433798d886a0672f4ea8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0190068188474cceba71c9e9913742b8" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 43375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:26:29.453898 16502 consensus_queue.cc:1035] T 9df2e12a84f94dcc8817974a9dd0b122 P 380ad4048218433798d886a0672f4ea8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67f3930d8c81496bad09ce3ed103df0a" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 38909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:26:29.457770 16017 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60076:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 07:26:29.457868 16017 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60076:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 07:26:29.458606 16017 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 51e2568956264fd9944bb4f1ac73c90d: Sending DeleteTablet for 3 replicas of tablet 9df2e12a84f94dcc8817974a9dd0b122
I20260925 07:26:29.458962 16412 tablet_service.cc:1514] Processing DeleteTablet for tablet 9df2e12a84f94dcc8817974a9dd0b122 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:26:29 UTC) from {username='kudu', principal='kudu/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:59488
I20260925 07:26:29.458978 16278 tablet_service.cc:1514] Processing DeleteTablet for tablet 9df2e12a84f94dcc8817974a9dd0b122 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:26:29 UTC) from {username='kudu', principal='kudu/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:50714
I20260925 07:26:29.458971 16144 tablet_service.cc:1514] Processing DeleteTablet for tablet 9df2e12a84f94dcc8817974a9dd0b122 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:26:29 UTC) from {username='kudu', principal='kudu/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:48054
I20260925 07:26:29.459244 16515 tablet_replica.cc:330] T 9df2e12a84f94dcc8817974a9dd0b122 P 380ad4048218433798d886a0672f4ea8: stopping tablet replica
I20260925 07:26:29.459347 16515 raft_consensus.cc:2229] T 9df2e12a84f94dcc8817974a9dd0b122 P 380ad4048218433798d886a0672f4ea8 [term 1 LEADER]: Raft consensus shutting down.
I20260925 07:26:29.459450 16515 raft_consensus.cc:2258] T 9df2e12a84f94dcc8817974a9dd0b122 P 380ad4048218433798d886a0672f4ea8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:26:29.459450 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 16081
I20260925 07:26:29.463083 16515 ts_tablet_manager.cc:1905] T 9df2e12a84f94dcc8817974a9dd0b122 P 380ad4048218433798d886a0672f4ea8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:26:29.464959 16515 ts_tablet_manager.cc:1918] T 9df2e12a84f94dcc8817974a9dd0b122 P 380ad4048218433798d886a0672f4ea8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 07:26:29.465026 16515 log.cc:1196] T 9df2e12a84f94dcc8817974a9dd0b122 P 380ad4048218433798d886a0672f4ea8: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/ts-2/wal/wals/9df2e12a84f94dcc8817974a9dd0b122
I20260925 07:26:29.465161 16517 tablet_replica.cc:330] T 9df2e12a84f94dcc8817974a9dd0b122 P 0190068188474cceba71c9e9913742b8: stopping tablet replica
I20260925 07:26:29.465301 16517 raft_consensus.cc:2229] T 9df2e12a84f94dcc8817974a9dd0b122 P 0190068188474cceba71c9e9913742b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 07:26:29.465330 16515 ts_tablet_manager.cc:1939] T 9df2e12a84f94dcc8817974a9dd0b122 P 380ad4048218433798d886a0672f4ea8: Deleting consensus metadata
I20260925 07:26:29.465458 16517 raft_consensus.cc:2258] T 9df2e12a84f94dcc8817974a9dd0b122 P 0190068188474cceba71c9e9913742b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:26:29.465664 16006 catalog_manager.cc:4841] TS 380ad4048218433798d886a0672f4ea8 (127.5.158.67:46263): tablet 9df2e12a84f94dcc8817974a9dd0b122 (table test-table [id=020006659ed1445a866d9e78b7005640]) successfully deleted
I20260925 07:26:29.465934 16517 ts_tablet_manager.cc:1905] T 9df2e12a84f94dcc8817974a9dd0b122 P 0190068188474cceba71c9e9913742b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:26:29.467787 16517 ts_tablet_manager.cc:1918] T 9df2e12a84f94dcc8817974a9dd0b122 P 0190068188474cceba71c9e9913742b8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 07:26:29.467859 16517 log.cc:1196] T 9df2e12a84f94dcc8817974a9dd0b122 P 0190068188474cceba71c9e9913742b8: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790321110323796-5753-0/minicluster-data/ts-1/wal/wals/9df2e12a84f94dcc8817974a9dd0b122
I20260925 07:26:29.468108 16517 ts_tablet_manager.cc:1939] T 9df2e12a84f94dcc8817974a9dd0b122 P 0190068188474cceba71c9e9913742b8: Deleting consensus metadata
I20260925 07:26:29.468542 16004 catalog_manager.cc:4841] TS 0190068188474cceba71c9e9913742b8 (127.5.158.66:43375): tablet 9df2e12a84f94dcc8817974a9dd0b122 (table test-table [id=020006659ed1445a866d9e78b7005640]) successfully deleted
W20260925 07:26:29.469111 16004 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.158.65:38909 (error 108)
I20260925 07:26:29.469290 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 16215
I20260925 07:26:29.470170 16490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:26:29.470034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:38909 (local address 127.0.0.1:48056)
0925 07:26:29.470097 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0925 07:26:29.470139 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.65:38909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260925 07:26:29.470323 16004 catalog_manager.cc:4584] TS 67f3930d8c81496bad09ce3ed103df0a (127.5.158.65:38909): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9df2e12a84f94dcc8817974a9dd0b122: Network error: Client connection negotiation failed: client connection to 127.5.158.65:38909: connect: Connection refused (error 111)
I20260925 07:26:29.473802 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 16349
I20260925 07:26:29.478416 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 15987
2026-09-25T07:26:29Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (1823 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 07:26:29 dist-test-slave-1drq krb5kdc[16521](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 07:26:29 dist-test-slave-1drq krb5kdc[16521](info): set up 2 sockets
Sep 25 07:26:29 dist-test-slave-1drq krb5kdc[16521](info): commencing operation
krb5kdc: starting...
W20260925 07:26:30.465391 5753 mini_kdc.cc:121] Time spent starting KDC: real 0.968s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Sep 25 07:26:30 dist-test-slave-1drq krb5kdc[16521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T07:26:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T07:26:30Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.5.158.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.5.158.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.5.158.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/krb5kdc/rangeradmin_127.5.158.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.5.158.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/krb5kdc/rangeradmin_127.5.158.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.5.158.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.5.158.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.5.158.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/krb5kdc/rangerlookup_127.5.158.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.5.158.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/krb5kdc/rangerlookup_127.5.158.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.158.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.5.158.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.158.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/krb5kdc/HTTP_127.5.158.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.158.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/krb5kdc/HTTP_127.5.158.84@KRBTEST.COM.keytab.
I20260925 07:26:30.623220 5753 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskROew7s/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/postgres -l logfile start
2026-09-25 07:26:32.854 UTC [16551] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-25 07:26:32.854 UTC [16551] LOG: listening on IPv4 address "127.5.158.84", port 43665
2026-09-25 07:26:32.858 UTC [16551] LOG: listening on Unix socket "/tmp/.s.PGSQL.43665"
2026-09-25 07:26:32.878 UTC [16554] LOG: database system was shut down at 2026-09-25 07:26:31 UTC
2026-09-25 07:26:32.882 UTC [16551] LOG: database system is ready to accept connections
I20260925 07:26:33.809114 5753 mini_postgres.cc:96] Postgres bound to 43665
2026-09-25 07:26:33.814 UTC [16562] FATAL: database "slave" does not exist
127.5.158.84:43665 - accepting connections
I20260925 07:26:33.815523 5753 mini_ranger.cc:161] Starting Ranger...
I20260925 07:26:33.830039 5753 mini_ranger.cc:84] Created miniranger Postgres user
I20260925 07:26:34.234551 5753 mini_ranger.cc:87] Created ranger Postgres database
I20260925 07:26:34.234613 5753 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ranger-admin
2026-09-25 07:26:34,486 [I] DB FLAVOR :POSTGRES
2026-09-25 07:26:34,486 [I] --------- Verifying Ranger DB connection ---------
2026-09-25 07:26:34,486 [I] Checking connection..
2026-09-25 07:26:34,486 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskROew7s/build/release/bin/postgresql.jar:/tmp/dist-test-taskROew7s/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.158.84:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-25 07:26:34,700 [I] Checking connection passed.
2026-09-25 07:26:34,701 [I] --------- Verifying version history table ---------
2026-09-25 07:26:34,701 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskROew7s/build/release/bin/postgresql.jar:/tmp/dist-test-taskROew7s/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.158.84:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 07:26:34,940 [I] Table x_db_version_h does not exist in database ranger
2026-09-25 07:26:34,941 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskROew7s/build/release/bin/postgresql.jar:/tmp/dist-test-taskROew7s/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.158.84:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 07:26:35,189 [I] Table x_db_version_h does not exist in database ranger
2026-09-25 07:26:35,190 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-25 07:26:35,190 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskROew7s/build/release/bin/postgresql.jar:/tmp/dist-test-taskROew7s/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.158.84:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskROew7s/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-25 07:26:35.414 UTC [16626] WARNING: there is no transaction in progress
2026-09-25 07:26:35,420 [I] create_dbversion_catalog.sql file imported successfully
2026-09-25 07:26:35,421 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskROew7s/build/release/bin/postgresql.jar:/tmp/dist-test-taskROew7s/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.158.84:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 07:26:35,630 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-25 07:26:35,630 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-25 07:26:35,630 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskROew7s/build/release/bin/postgresql.jar:/tmp/dist-test-taskROew7s/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.158.84:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-25 07:26:35,842 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskROew7s/build/release/bin/postgresql.jar:/tmp/dist-test-taskROew7s/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.158.84:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-25 07:26:36,047 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskROew7s/build/release/bin/postgresql.jar:/tmp/dist-test-taskROew7s/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.158.84:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-1drq.c.gcp-upstream.internal','N') ;"
2026-09-25 07:26:36,270 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskROew7s/build/release/bin/postgresql.jar:/tmp/dist-test-taskROew7s/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.158.84:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-25 07:26:36,505 [I] Table x_portal_user does not exist in database ranger
2026-09-25 07:26:36,505 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskROew7s/build/release/bin/postgresql.jar:/tmp/dist-test-taskROew7s/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.158.84:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-25 07:26:36,727 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-25 07:26:36,728 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskROew7s/build/release/bin/postgresql.jar:/tmp/dist-test-taskROew7s/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.158.84:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-25 07:26:36,988 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskROew7s/build/release/bin/postgresql.jar:/tmp/dist-test-taskROew7s/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.158.84:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-25 07:26:37,193 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-25 07:26:37,193 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskROew7s/build/release/bin/postgresql.jar:/tmp/dist-test-taskROew7s/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.158.84:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskROew7s/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-25 07:26:38.054 UTC [16734] WARNING: there is no transaction in progress
2026-09-25 07:26:38.069 UTC [16734] WARNING: there is no transaction in progress
2026-09-25 07:26:38.083 UTC [16734] WARNING: there is no transaction in progress
2026-09-25 07:26:38.095 UTC [16734] WARNING: there is no transaction in progress
2026-09-25 07:26:38.108 UTC [16734] WARNING: there is no transaction in progress
2026-09-25 07:26:38.124 UTC [16734] WARNING: there is no transaction in progress
2026-09-25 07:26:38.181 UTC [16734] WARNING: there is no transaction in progress
2026-09-25 07:26:38.187 UTC [16734] WARNING: there is no transaction in progress
2026-09-25 07:26:38.199 UTC [16734] WARNING: there is no transaction in progress
2026-09-25 07:26:38.208 UTC [16734] WARNING: there is no transaction in progress
2026-09-25 07:26:38.217 UTC [16734] WARNING: there is no transaction in progress
2026-09-25 07:26:38.229 UTC [16734] WARNING: there is no transaction in progress
2026-09-25 07:26:38.237 UTC [16734] WARNING: there is no transaction in progress
2026-09-25 07:26:38.242 UTC [16734] WARNING: there is no transaction in progress
2026-09-25 07:26:38.248 UTC [16734] WARNING: there is no transaction in progress
2026-09-25 07:26:38.253 UTC [16734] WARNING: there is no transaction in progress
2026-09-25 07:26:38,806 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-25 07:26:38,806 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskROew7s/build/release/bin/postgresql.jar:/tmp/dist-test-taskROew7s/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.158.84:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-25 07:26:39,031 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-25 07:26:39,031 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskROew7s/build/release/bin/postgresql.jar:/tmp/dist-test-taskROew7s/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.158.84:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-25 07:26:39,234 [I] Table x_portal_user already exists in database 'ranger'
2026-09-25 07:26:39,234 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskROew7s/build/release/bin/postgresql.jar:/tmp/dist-test-taskROew7s/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.158.84:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-25 07:26:39,473 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-25 07:26:39,473 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskROew7s/build/release/bin/postgresql.jar:/tmp/dist-test-taskROew7s/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.158.84:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-25 07:26:39,685 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskROew7s/build/release/bin/postgresql.jar:/tmp/dist-test-taskROew7s/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.158.84:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-25 07:26:39,914 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskROew7s/build/release/bin/postgresql.jar:/tmp/dist-test-taskROew7s/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.158.84:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-1drq.c.gcp-upstream.internal';"
2026-09-25 07:26:40,142 [I] CORE_DB_SCHEMA import status has been updated
2026-09-25 07:26:40,142 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskROew7s/build/release/bin/postgresql.jar:/tmp/dist-test-taskROew7s/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.158.84:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-25 07:26:40,383 [I] DB_PATCHES have already been applied
I20260925 07:26:40.388780 5753 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ranger-admin:/tmp/dist-test-taskROew7s/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskROew7s/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskROew7s/build/release/bin/postgresql.jar:/tmp/dist-test-taskROew7s/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260925 07:26:40.388850 5753 mini_ranger.cc:193] Using host: 127.5.158.84
I20260925 07:26:40.391695 5753 mini_ranger.cc:227] Ranger admin URL: http://127.5.158.84:55791
Sep 25, 2026 7:26:40 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 25, 2026 7:26:40 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 25, 2026 7:26:40 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ranger-admin/ews/webapp
Sep 25, 2026 7:26:40 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 25, 2026 7:26:40 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 25, 2026 7:26:41 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ranger-admin/ews/webapp].
Sep 25, 2026 7:26:41 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260925 07:26:41.469296 5753 mini_ranger.cc:160] Time spent starting Ranger: real 7.654s user 0.004s sys 0.004s
Sep 25, 2026 7:26:58 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 25, 2026 7:26:58 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Sep 25, 2026 7:26:58 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 25, 2026 7:26:58 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 25, 2026 7:26:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 25, 2026 7:26:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 25, 2026 7:26:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 25, 2026 7:26:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 25, 2026 7:26:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 25, 2026 7:26:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 25, 2026 7:26:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 25, 2026 7:26:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 25, 2026 7:26:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 25, 2026 7:26:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 25, 2026 7:26:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 25, 2026 7:26:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 25, 2026 7:26:58 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 25, 2026 7:26:59 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260925 07:27:00.028489 5753 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.5.158.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.5.158.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.5.158.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/krb5kdc/rangerkms_127.5.158.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.5.158.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/krb5kdc/rangerkms_127.5.158.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.158.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.5.158.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.158.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/krb5kdc/HTTP_127.5.158.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.158.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/krb5kdc/HTTP_127.5.158.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 25 07:27:00 dist-test-slave-1drq krb5kdc[16521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321220, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260925 07:27:00.100553 5753 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260925 07:27:00.108676 5753 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260925 07:27:00.842813 5753 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260925 07:27:00.842876 5753 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ranger-kms
I20260925 07:27:00.842888 5753 mini_ranger_kms.cc:227] Using postgres at 127.5.158.84:43665
2026-09-25 07:27:01,005 [I] DB FLAVOR :POSTGRES
2026-09-25 07:27:01,005 [I] --------- Verifying Ranger DB connection ---------
2026-09-25 07:27:01,006 [I] Checking connection
2026-09-25 07:27:01,006 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskROew7s/build/release/bin/postgresql.jar:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.158.84:43665/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-25 07:27:01,218 [I] connection success
2026-09-25 07:27:01,218 [I] --------- Verifying Ranger DB tables ---------
2026-09-25 07:27:01,218 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-25 07:27:01,218 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskROew7s/build/release/bin/postgresql.jar:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.158.84:43665/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-25 07:27:01,451 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-25 07:27:01,452 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-25 07:27:01,452 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-25 07:27:01,452 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskROew7s/build/release/bin/postgresql.jar:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.158.84:43665/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-25 07:27:01,686 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260925 07:27:02.092957 5753 mini_ranger_kms.cc:310] Created kms service
I20260925 07:27:02.253683 5753 mini_ranger_kms.cc:326] Created kudu user
I20260925 07:27:02.294170 5753 mini_ranger_kms.cc:343] Created rangerkms user
Sep 25 07:27:02 dist-test-slave-1drq krb5kdc[16521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321220, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.5.158.84@KRBTEST.COM
I20260925 07:27:02.680168 5753 mini_ranger_kms.cc:384] Added ranger policy
I20260925 07:27:02.680317 5753 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ranger-kms:/tmp/dist-test-taskROew7s/build/release/bin/postgresql.jar:/tmp/dist-test-taskROew7s/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskROew7s/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260925 07:27:02.680338 5753 mini_ranger_kms.cc:240] Using host: 127.5.158.84
I20260925 07:27:02.684188 5753 mini_ranger_kms.cc:276] Ranger KMS PID: 16914
I20260925 07:27:02.684305 5753 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.5.158.84:59609
Sep 25, 2026 7:27:03 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 25, 2026 7:27:03 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 25, 2026 7:27:03 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 25, 2026 7:27:03 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 25, 2026 7:27:03 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ranger-kms/ews/webapp].
Sep 25, 2026 7:27:03 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Sep 25 07:27:04 dist-test-slave-1drq krb5kdc[16521](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790321224, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.158.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:27:04.797842 5753 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 4.697s user 0.000s sys 0.147s
I20260925 07:27:04.797959 5753 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260925 07:27:04.798059 5753 mini_ranger_kms.cc:401] 127.5.158.84:59609/kms/v1/keys
Sep 25 07:27:05 dist-test-slave-1drq krb5kdc[16521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321224, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.158.84@KRBTEST.COM for HTTP/127.5.158.84@KRBTEST.COM
Sep 25 07:27:05 dist-test-slave-1drq krb5kdc[16521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321224, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.158.84@KRBTEST.COM for HTTP/127.5.158.84@KRBTEST.COM
Sep 25 07:27:07 dist-test-slave-1drq krb5kdc[16521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321227, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.5.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
I20260925 07:27:07.739401 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:33431
--webserver_interface=127.5.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:44771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.158.126:33431
--ranger_config_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.158.84:59609/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:27:07.835801 16965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:27:07.836062 16965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:27:07.836104 16965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:27:07.837639 16965 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:27:07.837714 16965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:27:07.837750 16965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:27:07.837785 16965 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:27:07.837814 16965 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:27:07.840004 16965 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:44771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/master-0/wal
--ranger_kms_url=127.5.158.84:59609/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.5.158.126:33431
--ranger_config_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:33431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16965
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:27:07.840231 16965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:27:07.840480 16965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:27:07.845474 16973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:27:07.845485 16970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:27:07.845696 16965 server_base.cc:1034] running on GCE node
W20260925 07:27:07.845530 16971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:27:07.846029 16965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:27:07.846303 16965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:27:07.847445 16965 hybrid_clock.cc:648] HybridClock initialized: now 1790321227847435 us; error 25 us; skew 500 ppm
Sep 25 07:27:07 dist-test-slave-1drq krb5kdc[16521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321227, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:27:07.855468 16965 init.cc:377] Logged in from keytab as kudu/127.5.158.126@KRBTEST.COM (short username kudu)
I20260925 07:27:07.856011 16965 webserver.cc:460] Webserver started at http://127.5.158.126:34917/ using document root <none> and password file <none>
I20260925 07:27:07.856282 16965 fs_manager.cc:359] Metadata directory not provided
I20260925 07:27:07.856336 16965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:27:07.856463 16965 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 07:27:07 dist-test-slave-1drq krb5kdc[16521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321227, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.126@KRBTEST.COM for HTTP/127.5.158.84@KRBTEST.COM
I20260925 07:27:07.908485 16965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/master-0/data/instance:
uuid: "c4cb2ae6904445e2b4201e476d4bfdad"
format_stamp: "Formatted at 2026-09-25 07:27:07 on dist-test-slave-1drq"
server_key: "65a1689abb7e4f7c72884d04efdc03b0"
server_key_iv: "550663b103e25441588e40c2e1188022"
server_key_version: "kuduclusterkey@0"
I20260925 07:27:07.908878 16965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/master-0/wal/instance:
uuid: "c4cb2ae6904445e2b4201e476d4bfdad"
format_stamp: "Formatted at 2026-09-25 07:27:07 on dist-test-slave-1drq"
server_key: "65a1689abb7e4f7c72884d04efdc03b0"
server_key_iv: "550663b103e25441588e40c2e1188022"
server_key_version: "kuduclusterkey@0"
I20260925 07:27:07.910476 16965 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260925 07:27:07.923170 16980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:07.923363 16965 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 07:27:07.923437 16965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "c4cb2ae6904445e2b4201e476d4bfdad"
format_stamp: "Formatted at 2026-09-25 07:27:07 on dist-test-slave-1drq"
server_key: "65a1689abb7e4f7c72884d04efdc03b0"
server_key_iv: "550663b103e25441588e40c2e1188022"
server_key_version: "kuduclusterkey@0"
I20260925 07:27:07.923501 16965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:27:07.949919 16965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:27:07.951711 16965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:27:07.951838 16965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:27:07.956661 16965 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.126:33431
I20260925 07:27:07.956682 17032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.126:33431 every 8 connection(s)
I20260925 07:27:07.957037 16965 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:27:07.957774 17033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:27:07.959931 17033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4cb2ae6904445e2b4201e476d4bfdad: Bootstrap starting.
I20260925 07:27:07.960440 17033 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c4cb2ae6904445e2b4201e476d4bfdad: Neither blocks nor log segments found. Creating new log.
I20260925 07:27:07.960687 17033 log.cc:824] T 00000000000000000000000000000000 P c4cb2ae6904445e2b4201e476d4bfdad: Log is configured to *not* fsync() on all Append() calls
I20260925 07:27:07.961297 17033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4cb2ae6904445e2b4201e476d4bfdad: No bootstrap required, opened a new log
I20260925 07:27:07.963100 17033 raft_consensus.cc:348] T 00000000000000000000000000000000 P c4cb2ae6904445e2b4201e476d4bfdad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4cb2ae6904445e2b4201e476d4bfdad" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 33431 } }
I20260925 07:27:07.963248 17033 raft_consensus.cc:374] T 00000000000000000000000000000000 P c4cb2ae6904445e2b4201e476d4bfdad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:27:07.963304 17033 raft_consensus.cc:729] T 00000000000000000000000000000000 P c4cb2ae6904445e2b4201e476d4bfdad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4cb2ae6904445e2b4201e476d4bfdad, State: Initialized, Role: FOLLOWER
I20260925 07:27:07.963454 17033 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4cb2ae6904445e2b4201e476d4bfdad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4cb2ae6904445e2b4201e476d4bfdad" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 33431 } }
I20260925 07:27:07.963542 17033 raft_consensus.cc:388] T 00000000000000000000000000000000 P c4cb2ae6904445e2b4201e476d4bfdad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 07:27:07.963593 17033 raft_consensus.cc:482] T 00000000000000000000000000000000 P c4cb2ae6904445e2b4201e476d4bfdad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 07:27:07.963655 17033 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4cb2ae6904445e2b4201e476d4bfdad [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:27:07.964162 17033 raft_consensus.cc:504] T 00000000000000000000000000000000 P c4cb2ae6904445e2b4201e476d4bfdad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4cb2ae6904445e2b4201e476d4bfdad" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 33431 } }
I20260925 07:27:07.964270 17033 leader_election.cc:302] T 00000000000000000000000000000000 P c4cb2ae6904445e2b4201e476d4bfdad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c4cb2ae6904445e2b4201e476d4bfdad; no voters:
I20260925 07:27:07.964445 17033 leader_election.cc:288] T 00000000000000000000000000000000 P c4cb2ae6904445e2b4201e476d4bfdad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 07:27:07.964468 17036 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c4cb2ae6904445e2b4201e476d4bfdad [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:27:07.964671 17036 raft_consensus.cc:686] T 00000000000000000000000000000000 P c4cb2ae6904445e2b4201e476d4bfdad [term 1 LEADER]: Becoming Leader. State: Replica: c4cb2ae6904445e2b4201e476d4bfdad, State: Running, Role: LEADER
I20260925 07:27:07.964807 17033 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4cb2ae6904445e2b4201e476d4bfdad [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:27:07.964823 17036 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4cb2ae6904445e2b4201e476d4bfdad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4cb2ae6904445e2b4201e476d4bfdad" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 33431 } }
I20260925 07:27:07.965253 17036 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4cb2ae6904445e2b4201e476d4bfdad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c4cb2ae6904445e2b4201e476d4bfdad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4cb2ae6904445e2b4201e476d4bfdad" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 33431 } } }
I20260925 07:27:07.965400 17036 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4cb2ae6904445e2b4201e476d4bfdad [sys.catalog]: This master's current role is: LEADER
I20260925 07:27:07.965561 17036 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4cb2ae6904445e2b4201e476d4bfdad [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4cb2ae6904445e2b4201e476d4bfdad. Latest consensus state: current_term: 1 leader_uuid: "c4cb2ae6904445e2b4201e476d4bfdad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4cb2ae6904445e2b4201e476d4bfdad" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 33431 } } }
I20260925 07:27:07.965651 17036 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4cb2ae6904445e2b4201e476d4bfdad [sys.catalog]: This master's current role is: LEADER
I20260925 07:27:07.965795 17045 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 07:27:07.966284 17045 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 07:27:07.966491 17033 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260925 07:27:07.966522 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 16965
I20260925 07:27:07.966598 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/master-0/wal/instance
I20260925 07:27:07.966775 5753 external_mini_cluster.cc:1428] Setting key 4f8b42b09154655658a2672ec5f6299a
I20260925 07:27:07.973294 17045 catalog_manager.cc:1347] Generated new cluster ID: 7ee2cedd86544368ad5f35615006ce6f
I20260925 07:27:07.973395 17045 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 07:27:07 dist-test-slave-1drq krb5kdc[16521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321227, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:27:07.987808 17055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:07.967665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51896 (local address 127.5.158.126:33431)
0925 07:27:07.973269 (+ 5604us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:07.973280 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:07.977246 (+ 3966us) server_negotiation.cc:407] Connection header received
0925 07:27:07.977419 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:07.977425 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:07.977475 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:07.977593 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:27:07.978209 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:07.979850 (+ 1641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:07.981016 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:07.981681 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:07.981785 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:07.984538 (+ 2753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:27:07.984542 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:27:07.984542 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:27:07.984553 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:27:07.986411 (+ 1858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:07.986731 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:07.986733 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:07.986733 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:07.986818 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:07.986980 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:07.986981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:07.986981 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:07.987264 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:27:07.987416 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:07.987476 (+ 60us) server_negotiation.cc:299] Negotiation successful
0925 07:27:07.987595 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5485,"thread_start_us":86,"threads_started":1}
I20260925 07:27:07.990748 17045 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 07:27:07.991400 17045 catalog_manager.cc:1504] Loading token signing keys...
I20260925 07:27:08.000473 17045 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c4cb2ae6904445e2b4201e476d4bfdad: Generated new TSK 0
I20260925 07:27:08.000705 17045 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.158.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab.
Entry for principal kudu/127.5.158.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab.
I20260925 07:27:08.702535 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.158.84:59609/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.65:0
--local_ip_for_outbound_sockets=127.5.158.65
--webserver_interface=127.5.158.65
--webserver_port=0
--tserver_master_addrs=127.5.158.126:33431
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab
--principal=kudu/127.5.158.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:44771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:27:08.815644 17078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:27:08.815898 17078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:27:08.815941 17078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:27:08.817554 17078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:27:08.817615 17078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:27:08.817656 17078 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.65
I20260925 07:27:08.819590 17078 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:44771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.5.158.84:59609/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab
--principal=kudu/127.5.158.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.158.65
--webserver_port=0
--tserver_master_addrs=127.5.158.126:33431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17078
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.65
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:27:08.819846 17078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:27:08.820093 17078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:27:08.825259 17086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:27:08.825259 17084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:27:08.825276 17078 server_base.cc:1034] running on GCE node
W20260925 07:27:08.825240 17083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:27:08.825713 17078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:27:08.825939 17078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:27:08.827126 17078 hybrid_clock.cc:648] HybridClock initialized: now 1790321228827113 us; error 30 us; skew 500 ppm
Sep 25 07:27:08 dist-test-slave-1drq krb5kdc[16521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321228, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:27:08.829730 17078 init.cc:377] Logged in from keytab as kudu/127.5.158.65@KRBTEST.COM (short username kudu)
I20260925 07:27:08.830133 17078 webserver.cc:460] Webserver started at http://127.5.158.65:40163/ using document root <none> and password file <none>
I20260925 07:27:08.830336 17078 fs_manager.cc:359] Metadata directory not provided
I20260925 07:27:08.830402 17078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:27:08.830525 17078 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 07:27:08 dist-test-slave-1drq krb5kdc[16521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321228, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.65@KRBTEST.COM for HTTP/127.5.158.84@KRBTEST.COM
I20260925 07:27:08.841981 17078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ts-0/data/instance:
uuid: "131634a7a8274f75b01beb82e3850c60"
format_stamp: "Formatted at 2026-09-25 07:27:08 on dist-test-slave-1drq"
server_key: "718f0a9cefed4396c7b475d979eaa15f"
server_key_iv: "e05ca0b04f1a06d8a2f6655c8b53b68b"
server_key_version: "kuduclusterkey@0"
I20260925 07:27:08.842409 17078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ts-0/wal/instance:
uuid: "131634a7a8274f75b01beb82e3850c60"
format_stamp: "Formatted at 2026-09-25 07:27:08 on dist-test-slave-1drq"
server_key: "718f0a9cefed4396c7b475d979eaa15f"
server_key_iv: "e05ca0b04f1a06d8a2f6655c8b53b68b"
server_key_version: "kuduclusterkey@0"
I20260925 07:27:08.845268 17078 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.000s
I20260925 07:27:08.856896 17093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:08.857097 17078 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 07:27:08.857213 17078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ts-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ts-0/wal
uuid: "131634a7a8274f75b01beb82e3850c60"
format_stamp: "Formatted at 2026-09-25 07:27:08 on dist-test-slave-1drq"
server_key: "718f0a9cefed4396c7b475d979eaa15f"
server_key_iv: "e05ca0b04f1a06d8a2f6655c8b53b68b"
server_key_version: "kuduclusterkey@0"
I20260925 07:27:08.857288 17078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:27:08.909078 17078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:27:08.911491 17078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:27:08.911643 17078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:27:08.911897 17078 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:27:08.912267 17078 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:27:08.912317 17078 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:08.912348 17078 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:27:08.912379 17078 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:08.920014 17078 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.65:41407
I20260925 07:27:08.920135 17208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.65:41407 every 8 connection(s)
I20260925 07:27:08.920492 17078 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
Sep 25 07:27:08 dist-test-slave-1drq krb5kdc[16521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321228, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.65@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:27:08.930051 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 17078
I20260925 07:27:08.930171 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ts-0/wal/instance
I20260925 07:27:08.930394 5753 external_mini_cluster.cc:1428] Setting key 5ba520b6c5c769bced9e5ff353c08b75
I20260925 07:27:08.932271 17211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:08.921196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:36669 (local address 127.5.158.126:33431)
0925 07:27:08.921342 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:08.921344 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:08.921593 (+ 249us) server_negotiation.cc:407] Connection header received
0925 07:27:08.922193 (+ 600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:08.922194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:08.922206 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:08.922284 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:27:08.922937 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:08.923792 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:08.925323 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:08.925648 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:08.925721 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:08.928556 (+ 2835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:27:08.928558 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:27:08.928558 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:27:08.928565 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:27:08.930314 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:08.931314 (+ 1000us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:08.931315 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:08.931316 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:08.931379 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:08.931619 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:08.931620 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:08.931620 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:08.931733 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:27:08.931789 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:08.932102 (+ 313us) server_negotiation.cc:299] Negotiation successful
0925 07:27:08.932183 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":97,"threads_started":1}
I20260925 07:27:08.932420 17212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:08.921197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:33431 (local address 127.5.158.65:36669)
0925 07:27:08.921474 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:08.921512 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 07:27:08.922075 (+ 563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:27:08.922369 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:27:08.922374 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:27:08.922589 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:27:08.922839 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:08.922846 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:08.923903 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:08.923904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:08.925232 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:08.925235 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:08.925745 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:08.925746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:08.925887 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:08.926422 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:27:08.926432 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:27:08.928456 (+ 2024us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:27:08.930414 (+ 1958us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:27:08.930420 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:27:08.930422 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:27:08.931235 (+ 813us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:27:08.931445 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:27:08.931445 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:27:08.931446 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:27:08.931559 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:27:08.931812 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:27:08.931816 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:27:08.931966 (+ 150us) client_negotiation.cc:769] Sending connection context
0925 07:27:08.932080 (+ 114us) client_negotiation.cc:240] Negotiation successful
0925 07:27:08.932201 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":101,"threads_started":1}
I20260925 07:27:08.933018 17209 heartbeater.cc:347] Connected to a master server at 127.5.158.126:33431
I20260925 07:27:08.933131 17209 heartbeater.cc:464] Registering TS with master...
I20260925 07:27:08.933308 17209 heartbeater.cc:511] Master 127.5.158.126:33431 requested a full tablet report, sending...
I20260925 07:27:08.933846 16997 ts_manager.cc:194] Registered new tserver with Master: 131634a7a8274f75b01beb82e3850c60 (127.5.158.65:41407)
I20260925 07:27:08.934677 16997 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.65@KRBTEST.COM'} at 127.5.158.65:36669
WARNING: no policy specified for kudu/127.5.158.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab.
Entry for principal kudu/127.5.158.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab.
I20260925 07:27:08.968073 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.158.84:59609/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.66:0
--local_ip_for_outbound_sockets=127.5.158.66
--webserver_interface=127.5.158.66
--webserver_port=0
--tserver_master_addrs=127.5.158.126:33431
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab
--principal=kudu/127.5.158.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:44771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 07:27:09 dist-test-slave-1drq krb5kdc[16521](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790321229, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260925 07:27:09.063114 17217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:27:09.063352 17217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:27:09.063390 17217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:27:09.065030 17217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:27:09.065174 17217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:27:09.065264 17217 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.66
I20260925 07:27:09.067363 17217 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:44771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.5.158.84:59609/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab
--principal=kudu/127.5.158.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.158.66
--webserver_port=0
--tserver_master_addrs=127.5.158.126:33431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17217
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.66
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:27:09.067790 17217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:27:09.068115 17217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:27:09.073824 17222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:27:09.073833 17225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:27:09.074095 17223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:27:09.074795 17217 server_base.cc:1034] running on GCE node
I20260925 07:27:09.075011 17217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:27:09.075232 17217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:27:09.077164 17217 hybrid_clock.cc:648] HybridClock initialized: now 1790321229077140 us; error 39 us; skew 500 ppm
Sep 25 07:27:09 dist-test-slave-1drq krb5kdc[16521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321229, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:27:09.080051 17217 init.cc:377] Logged in from keytab as kudu/127.5.158.66@KRBTEST.COM (short username kudu)
I20260925 07:27:09.080546 17217 webserver.cc:460] Webserver started at http://127.5.158.66:40357/ using document root <none> and password file <none>
I20260925 07:27:09.080760 17217 fs_manager.cc:359] Metadata directory not provided
I20260925 07:27:09.080813 17217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:27:09.080977 17217 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 07:27:09 dist-test-slave-1drq krb5kdc[16521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321229, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.66@KRBTEST.COM for HTTP/127.5.158.84@KRBTEST.COM
I20260925 07:27:09.092353 17217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ts-1/data/instance:
uuid: "279b2843fb1240e3bec794ac0b04bfb2"
format_stamp: "Formatted at 2026-09-25 07:27:09 on dist-test-slave-1drq"
server_key: "ab06d2e45fac9bade929848448bb5b28"
server_key_iv: "1e1398c2c00c503af6216f998e751b54"
server_key_version: "kuduclusterkey@0"
I20260925 07:27:09.092855 17217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ts-1/wal/instance:
uuid: "279b2843fb1240e3bec794ac0b04bfb2"
format_stamp: "Formatted at 2026-09-25 07:27:09 on dist-test-slave-1drq"
server_key: "ab06d2e45fac9bade929848448bb5b28"
server_key_iv: "1e1398c2c00c503af6216f998e751b54"
server_key_version: "kuduclusterkey@0"
I20260925 07:27:09.094738 17217 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260925 07:27:09.105559 17232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:09.105767 17217 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 07:27:09.105849 17217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ts-1/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ts-1/wal
uuid: "279b2843fb1240e3bec794ac0b04bfb2"
format_stamp: "Formatted at 2026-09-25 07:27:09 on dist-test-slave-1drq"
server_key: "ab06d2e45fac9bade929848448bb5b28"
server_key_iv: "1e1398c2c00c503af6216f998e751b54"
server_key_version: "kuduclusterkey@0"
I20260925 07:27:09.105923 17217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:27:09.118542 17217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:27:09.120407 17217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:27:09.120600 17217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:27:09.120879 17217 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:27:09.121235 17217 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:27:09.121294 17217 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:09.121354 17217 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:27:09.121380 17217 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:09.128252 17217 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.66:41809
I20260925 07:27:09.128618 17217 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
I20260925 07:27:09.129083 17346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.66:41809 every 8 connection(s)
I20260925 07:27:09.129706 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 17217
I20260925 07:27:09.129810 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ts-1/wal/instance
I20260925 07:27:09.130000 5753 external_mini_cluster.cc:1428] Setting key 812cf8ce7586b187c303aeae62917102
Sep 25 07:27:09 dist-test-slave-1drq krb5kdc[16521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321229, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.66@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:27:09.138581 17211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:09.130204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:57921 (local address 127.5.158.126:33431)
0925 07:27:09.130297 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:09.130299 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:09.130305 (+ 6us) server_negotiation.cc:407] Connection header received
0925 07:27:09.130326 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:09.130327 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:09.130337 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:09.130387 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:27:09.131501 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:09.132225 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:09.133758 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:09.133998 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:09.134031 (+ 33us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:09.136080 (+ 2049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:27:09.136082 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:27:09.136082 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:27:09.136089 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:27:09.137344 (+ 1255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:09.137706 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:09.137707 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:09.137707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:09.137759 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:09.137990 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:09.137991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:09.137991 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:09.138088 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:27:09.138150 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:09.138394 (+ 244us) server_negotiation.cc:299] Negotiation successful
0925 07:27:09.138481 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260925 07:27:09.138847 17349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:09.129297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:33431 (local address 127.5.158.66:57921)
0925 07:27:09.129423 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:09.129470 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0925 07:27:09.130101 (+ 631us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:27:09.130464 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:27:09.130468 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:27:09.130681 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:27:09.131378 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:09.131384 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:09.132335 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:09.132336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:09.133631 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:09.133635 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:09.134133 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:09.134134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:09.134261 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:09.134794 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:27:09.134803 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:27:09.135997 (+ 1194us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:27:09.137429 (+ 1432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:27:09.137435 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:27:09.137436 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:27:09.137626 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:27:09.137815 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:27:09.137816 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:27:09.137816 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:27:09.137916 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:27:09.138153 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:27:09.138159 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:27:09.138275 (+ 116us) client_negotiation.cc:769] Sending connection context
0925 07:27:09.138362 (+ 87us) client_negotiation.cc:240] Negotiation successful
0925 07:27:09.138557 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":71,"threads_started":1}
I20260925 07:27:09.139243 17347 heartbeater.cc:347] Connected to a master server at 127.5.158.126:33431
I20260925 07:27:09.139325 17347 heartbeater.cc:464] Registering TS with master...
I20260925 07:27:09.139487 17347 heartbeater.cc:511] Master 127.5.158.126:33431 requested a full tablet report, sending...
I20260925 07:27:09.139839 16997 ts_manager.cc:194] Registered new tserver with Master: 279b2843fb1240e3bec794ac0b04bfb2 (127.5.158.66:41809)
I20260925 07:27:09.140194 16997 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.66@KRBTEST.COM'} at 127.5.158.66:57921
WARNING: no policy specified for kudu/127.5.158.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab.
Entry for principal kudu/127.5.158.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab.
I20260925 07:27:09.160970 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.158.84:59609/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.67:0
--local_ip_for_outbound_sockets=127.5.158.67
--webserver_interface=127.5.158.67
--webserver_port=0
--tserver_master_addrs=127.5.158.126:33431
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab
--principal=kudu/127.5.158.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:44771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:27:09.245199 17352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:27:09.245393 17352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:27:09.245426 17352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:27:09.246515 17352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:27:09.246574 17352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:27:09.246616 17352 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.67
I20260925 07:27:09.247900 17352 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:44771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.5.158.84:59609/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab
--principal=kudu/127.5.158.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.158.67
--webserver_port=0
--tserver_master_addrs=127.5.158.126:33431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.67
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:27:09.248131 17352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:27:09.248330 17352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:27:09.252830 17358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:27:09.253141 17360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:27:09.252830 17357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:27:09.254002 17352 server_base.cc:1034] running on GCE node
I20260925 07:27:09.254197 17352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:27:09.254455 17352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:27:09.255601 17352 hybrid_clock.cc:648] HybridClock initialized: now 1790321229255594 us; error 32 us; skew 500 ppm
Sep 25 07:27:09 dist-test-slave-1drq krb5kdc[16521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321229, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:27:09.258294 17352 init.cc:377] Logged in from keytab as kudu/127.5.158.67@KRBTEST.COM (short username kudu)
I20260925 07:27:09.258728 17352 webserver.cc:460] Webserver started at http://127.5.158.67:36727/ using document root <none> and password file <none>
I20260925 07:27:09.258966 17352 fs_manager.cc:359] Metadata directory not provided
I20260925 07:27:09.259029 17352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:27:09.259160 17352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 07:27:09 dist-test-slave-1drq krb5kdc[16521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321229, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.67@KRBTEST.COM for HTTP/127.5.158.84@KRBTEST.COM
I20260925 07:27:09.271724 17352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ts-2/data/instance:
uuid: "3da63b99fc5f49cca9e692bb00865238"
format_stamp: "Formatted at 2026-09-25 07:27:09 on dist-test-slave-1drq"
server_key: "f0b30b16900d2cc4a9a6e50d22d3376b"
server_key_iv: "9a4befc3f0f2ee96e377217dfde5a891"
server_key_version: "kuduclusterkey@0"
I20260925 07:27:09.272238 17352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ts-2/wal/instance:
uuid: "3da63b99fc5f49cca9e692bb00865238"
format_stamp: "Formatted at 2026-09-25 07:27:09 on dist-test-slave-1drq"
server_key: "f0b30b16900d2cc4a9a6e50d22d3376b"
server_key_iv: "9a4befc3f0f2ee96e377217dfde5a891"
server_key_version: "kuduclusterkey@0"
I20260925 07:27:09.281441 17352 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.000s sys 0.001s
I20260925 07:27:09.293191 17367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:09.293337 17352 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 07:27:09.293429 17352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ts-2/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ts-2/wal
uuid: "3da63b99fc5f49cca9e692bb00865238"
format_stamp: "Formatted at 2026-09-25 07:27:09 on dist-test-slave-1drq"
server_key: "f0b30b16900d2cc4a9a6e50d22d3376b"
server_key_iv: "9a4befc3f0f2ee96e377217dfde5a891"
server_key_version: "kuduclusterkey@0"
I20260925 07:27:09.293516 17352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:27:09.310578 17352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:27:09.312664 17352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:27:09.312788 17352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:27:09.312958 17352 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:27:09.313282 17352 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:27:09.313329 17352 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:09.313370 17352 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:27:09.313388 17352 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:09.320287 17352 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.67:40527
I20260925 07:27:09.320672 17352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
I20260925 07:27:09.322896 17480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.67:40527 every 8 connection(s)
I20260925 07:27:09.327714 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 17352
I20260925 07:27:09.327827 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ts-2/wal/instance
I20260925 07:27:09.328033 5753 external_mini_cluster.cc:1428] Setting key da99213cba2706ee838ccf2708f91d41
Sep 25 07:27:09 dist-test-slave-1drq krb5kdc[16521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321229, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.67@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:27:09.332239 17211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:09.321379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:55341 (local address 127.5.158.126:33431)
0925 07:27:09.321427 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:09.321429 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:09.323195 (+ 1766us) server_negotiation.cc:407] Connection header received
0925 07:27:09.323825 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:09.323826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:09.323840 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:09.323931 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:27:09.324623 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:09.325414 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:09.326818 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:09.327097 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:09.327192 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:09.329508 (+ 2316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:27:09.329510 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:27:09.329510 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:27:09.329517 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:27:09.330888 (+ 1371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:09.331320 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:09.331321 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:09.331321 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:09.331381 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:09.331626 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:09.331627 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:09.331627 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:09.331735 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:27:09.331823 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:09.332070 (+ 247us) server_negotiation.cc:299] Negotiation successful
0925 07:27:09.332151 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 07:27:09.332398 17483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:09.321352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:33431 (local address 127.5.158.67:55341)
0925 07:27:09.323061 (+ 1709us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:09.323098 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 07:27:09.323702 (+ 604us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:27:09.323998 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:27:09.324002 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:27:09.324242 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:27:09.324503 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:09.324509 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:09.325520 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:09.325521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:09.326713 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:09.326716 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:09.327251 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:09.327252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:09.327351 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:09.327891 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:27:09.327901 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:27:09.329427 (+ 1526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:27:09.330989 (+ 1562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:27:09.330996 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:27:09.330998 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:27:09.331215 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:27:09.331454 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:27:09.331454 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:27:09.331455 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:27:09.331561 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:27:09.331815 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:27:09.331821 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:27:09.331949 (+ 128us) client_negotiation.cc:769] Sending connection context
0925 07:27:09.332051 (+ 102us) client_negotiation.cc:240] Negotiation successful
0925 07:27:09.332177 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1673,"thread_start_us":660,"threads_started":1}
I20260925 07:27:09.332809 17481 heartbeater.cc:347] Connected to a master server at 127.5.158.126:33431
I20260925 07:27:09.332893 17481 heartbeater.cc:464] Registering TS with master...
I20260925 07:27:09.333069 17481 heartbeater.cc:511] Master 127.5.158.126:33431 requested a full tablet report, sending...
I20260925 07:27:09.333483 16997 ts_manager.cc:194] Registered new tserver with Master: 3da63b99fc5f49cca9e692bb00865238 (127.5.158.67:40527)
I20260925 07:27:09.333968 16997 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.67@KRBTEST.COM'} at 127.5.158.67:55341
I20260925 07:27:09.340116 5753 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 07:27:09.348764 17211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:09.341457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51912 (local address 127.5.158.126:33431)
0925 07:27:09.341502 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:09.341505 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:09.341629 (+ 124us) server_negotiation.cc:407] Connection header received
0925 07:27:09.341697 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:09.341698 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:09.341706 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:09.341752 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:27:09.342424 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:09.342870 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:09.343905 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:09.344081 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:09.344140 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:09.345995 (+ 1855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:27:09.345996 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:27:09.345996 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:27:09.346002 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:27:09.347317 (+ 1315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:09.348077 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:09.348078 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:09.348078 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:09.348124 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:09.348341 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:09.348342 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:09.348342 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:09.348430 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:27:09.348500 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:09.348658 (+ 158us) server_negotiation.cc:299] Negotiation successful
0925 07:27:09.348708 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 07:27:09.350610 16997 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51912:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 07:27:09.360351 16997 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 07:27:09.368040 17493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:09.363105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.66:41809 (local address 127.0.0.1:33840)
0925 07:27:09.363528 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:09.363564 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 07:27:09.363626 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:27:09.363998 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:27:09.364001 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:27:09.364003 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:27:09.364063 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:09.364065 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:09.365376 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:09.365377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:09.366709 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:09.366712 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:09.367770 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:09.367771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:09.367846 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:09.367860 (+ 14us) client_negotiation.cc:769] Sending connection context
0925 07:27:09.367925 (+ 65us) client_negotiation.cc:240] Negotiation successful
0925 07:27:09.367968 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":72,"threads_started":1}
I20260925 07:27:09.368371 17496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:09.363360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33840 (local address 127.5.158.66:41809)
0925 07:27:09.363779 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:09.363784 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:09.363793 (+ 9us) server_negotiation.cc:407] Connection header received
0925 07:27:09.363834 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:09.363836 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:09.363852 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:09.363922 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:27:09.364140 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:09.365270 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:09.366828 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:09.367600 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:09.367669 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:09.367737 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:09.368233 (+ 496us) server_negotiation.cc:299] Negotiation successful
0925 07:27:09.368275 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":80,"threads_started":1}
I20260925 07:27:09.368952 17281 tablet_service.cc:1467] Processing CreateTablet for tablet 54bfdbcc0d184e9ea3450858d7c79dfa (DEFAULT_TABLE table=test-table [id=68a5abc9b63f4c8fba443cf211f48040]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:27:09.369315 17281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54bfdbcc0d184e9ea3450858d7c79dfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:27:09.372097 17494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:09.363562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:41407 (local address 127.0.0.1:56772)
0925 07:27:09.364659 (+ 1097us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:09.364668 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 07:27:09.364713 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:27:09.365372 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:27:09.365372 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:27:09.365374 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:27:09.365408 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:09.365409 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:09.367027 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:09.367028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:09.370869 (+ 3841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:09.370871 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:09.371860 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:09.371861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:09.371916 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:09.371920 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 07:27:09.371985 (+ 65us) client_negotiation.cc:240] Negotiation successful
0925 07:27:09.372022 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1084,"mutex_wait_us":4,"thread_start_us":90,"threads_started":1}
I20260925 07:27:09.373070 17499 tablet_bootstrap.cc:492] T 54bfdbcc0d184e9ea3450858d7c79dfa P 279b2843fb1240e3bec794ac0b04bfb2: Bootstrap starting.
I20260925 07:27:09.373745 17497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:09.363106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.67:40527 (local address 127.0.0.1:39472)
0925 07:27:09.364451 (+ 1345us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:09.364461 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 07:27:09.364504 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:27:09.365246 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:27:09.365247 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:27:09.365249 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:27:09.365286 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:09.365288 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:09.369011 (+ 3723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:09.369012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:09.370373 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:09.370376 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:09.373502 (+ 3126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:09.373504 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:09.373563 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:09.373567 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 07:27:09.373623 (+ 56us) client_negotiation.cc:240] Negotiation successful
0925 07:27:09.373677 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1329,"mutex_wait_us":2,"thread_start_us":92,"threads_started":1}
I20260925 07:27:09.374046 17498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:09.363766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56772 (local address 127.5.158.65:41407)
0925 07:27:09.364612 (+ 846us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:09.364616 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:09.364762 (+ 146us) server_negotiation.cc:407] Connection header received
0925 07:27:09.364803 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:09.364805 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:09.364820 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:09.364888 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:27:09.365479 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:09.366919 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:09.370977 (+ 4058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:09.371695 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:09.371763 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:09.371827 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:09.373926 (+ 2099us) server_negotiation.cc:299] Negotiation successful
0925 07:27:09.373971 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":816,"thread_start_us":44,"threads_started":1}
I20260925 07:27:09.374073 17499 tablet_bootstrap.cc:654] T 54bfdbcc0d184e9ea3450858d7c79dfa P 279b2843fb1240e3bec794ac0b04bfb2: Neither blocks nor log segments found. Creating new log.
I20260925 07:27:09.374346 17499 log.cc:824] T 54bfdbcc0d184e9ea3450858d7c79dfa P 279b2843fb1240e3bec794ac0b04bfb2: Log is configured to *not* fsync() on all Append() calls
I20260925 07:27:09.374616 17143 tablet_service.cc:1467] Processing CreateTablet for tablet 54bfdbcc0d184e9ea3450858d7c79dfa (DEFAULT_TABLE table=test-table [id=68a5abc9b63f4c8fba443cf211f48040]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:27:09.374935 17143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54bfdbcc0d184e9ea3450858d7c79dfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:27:09.375026 17499 tablet_bootstrap.cc:492] T 54bfdbcc0d184e9ea3450858d7c79dfa P 279b2843fb1240e3bec794ac0b04bfb2: No bootstrap required, opened a new log
I20260925 07:27:09.375092 17499 ts_tablet_manager.cc:1397] T 54bfdbcc0d184e9ea3450858d7c79dfa P 279b2843fb1240e3bec794ac0b04bfb2: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260925 07:27:09.376788 17499 raft_consensus.cc:348] T 54bfdbcc0d184e9ea3450858d7c79dfa P 279b2843fb1240e3bec794ac0b04bfb2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "279b2843fb1240e3bec794ac0b04bfb2" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 41809 } } peers { permanent_uuid: "3da63b99fc5f49cca9e692bb00865238" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 40527 } } peers { permanent_uuid: "131634a7a8274f75b01beb82e3850c60" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41407 } }
I20260925 07:27:09.376953 17499 raft_consensus.cc:374] T 54bfdbcc0d184e9ea3450858d7c79dfa P 279b2843fb1240e3bec794ac0b04bfb2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:27:09.376996 17499 raft_consensus.cc:729] T 54bfdbcc0d184e9ea3450858d7c79dfa P 279b2843fb1240e3bec794ac0b04bfb2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 279b2843fb1240e3bec794ac0b04bfb2, State: Initialized, Role: FOLLOWER
I20260925 07:27:09.377120 17499 consensus_queue.cc:260] T 54bfdbcc0d184e9ea3450858d7c79dfa P 279b2843fb1240e3bec794ac0b04bfb2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "279b2843fb1240e3bec794ac0b04bfb2" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 41809 } } peers { permanent_uuid: "3da63b99fc5f49cca9e692bb00865238" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 40527 } } peers { permanent_uuid: "131634a7a8274f75b01beb82e3850c60" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41407 } }
I20260925 07:27:09.377363 17499 ts_tablet_manager.cc:1428] T 54bfdbcc0d184e9ea3450858d7c79dfa P 279b2843fb1240e3bec794ac0b04bfb2: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260925 07:27:09.377480 17347 heartbeater.cc:503] Master 127.5.158.126:33431 was elected leader, sending a full tablet report...
I20260925 07:27:09.378477 17495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:09.363358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39472 (local address 127.5.158.67:40527)
0925 07:27:09.365017 (+ 1659us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:09.365021 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:09.365030 (+ 9us) server_negotiation.cc:407] Connection header received
0925 07:27:09.365064 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:09.365066 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:09.365100 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:09.365171 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:27:09.367901 (+ 2730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:09.368919 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:09.372260 (+ 3341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:09.373337 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:09.373401 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:09.373469 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:09.378355 (+ 4886us) server_negotiation.cc:299] Negotiation successful
0925 07:27:09.378395 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1623,"thread_start_us":70,"threads_started":1}
I20260925 07:27:09.379072 17415 tablet_service.cc:1467] Processing CreateTablet for tablet 54bfdbcc0d184e9ea3450858d7c79dfa (DEFAULT_TABLE table=test-table [id=68a5abc9b63f4c8fba443cf211f48040]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:27:09.379370 17415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54bfdbcc0d184e9ea3450858d7c79dfa. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 07:27:09.379421 17348 tablet.cc:2369] T 54bfdbcc0d184e9ea3450858d7c79dfa P 279b2843fb1240e3bec794ac0b04bfb2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 07:27:09.381264 17502 tablet_bootstrap.cc:492] T 54bfdbcc0d184e9ea3450858d7c79dfa P 131634a7a8274f75b01beb82e3850c60: Bootstrap starting.
I20260925 07:27:09.382197 17503 tablet_bootstrap.cc:492] T 54bfdbcc0d184e9ea3450858d7c79dfa P 3da63b99fc5f49cca9e692bb00865238: Bootstrap starting.
I20260925 07:27:09.382268 17502 tablet_bootstrap.cc:654] T 54bfdbcc0d184e9ea3450858d7c79dfa P 131634a7a8274f75b01beb82e3850c60: Neither blocks nor log segments found. Creating new log.
I20260925 07:27:09.382576 17502 log.cc:824] T 54bfdbcc0d184e9ea3450858d7c79dfa P 131634a7a8274f75b01beb82e3850c60: Log is configured to *not* fsync() on all Append() calls
I20260925 07:27:09.383177 17503 tablet_bootstrap.cc:654] T 54bfdbcc0d184e9ea3450858d7c79dfa P 3da63b99fc5f49cca9e692bb00865238: Neither blocks nor log segments found. Creating new log.
I20260925 07:27:09.383358 17502 tablet_bootstrap.cc:492] T 54bfdbcc0d184e9ea3450858d7c79dfa P 131634a7a8274f75b01beb82e3850c60: No bootstrap required, opened a new log
I20260925 07:27:09.383425 17502 ts_tablet_manager.cc:1397] T 54bfdbcc0d184e9ea3450858d7c79dfa P 131634a7a8274f75b01beb82e3850c60: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 07:27:09.383477 17503 log.cc:824] T 54bfdbcc0d184e9ea3450858d7c79dfa P 3da63b99fc5f49cca9e692bb00865238: Log is configured to *not* fsync() on all Append() calls
I20260925 07:27:09.384101 17503 tablet_bootstrap.cc:492] T 54bfdbcc0d184e9ea3450858d7c79dfa P 3da63b99fc5f49cca9e692bb00865238: No bootstrap required, opened a new log
I20260925 07:27:09.384172 17503 ts_tablet_manager.cc:1397] T 54bfdbcc0d184e9ea3450858d7c79dfa P 3da63b99fc5f49cca9e692bb00865238: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 07:27:09.385646 17503 raft_consensus.cc:348] T 54bfdbcc0d184e9ea3450858d7c79dfa P 3da63b99fc5f49cca9e692bb00865238 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "279b2843fb1240e3bec794ac0b04bfb2" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 41809 } } peers { permanent_uuid: "3da63b99fc5f49cca9e692bb00865238" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 40527 } } peers { permanent_uuid: "131634a7a8274f75b01beb82e3850c60" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41407 } }
I20260925 07:27:09.385807 17503 raft_consensus.cc:374] T 54bfdbcc0d184e9ea3450858d7c79dfa P 3da63b99fc5f49cca9e692bb00865238 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:27:09.385766 17502 raft_consensus.cc:348] T 54bfdbcc0d184e9ea3450858d7c79dfa P 131634a7a8274f75b01beb82e3850c60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "279b2843fb1240e3bec794ac0b04bfb2" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 41809 } } peers { permanent_uuid: "3da63b99fc5f49cca9e692bb00865238" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 40527 } } peers { permanent_uuid: "131634a7a8274f75b01beb82e3850c60" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41407 } }
I20260925 07:27:09.385867 17503 raft_consensus.cc:729] T 54bfdbcc0d184e9ea3450858d7c79dfa P 3da63b99fc5f49cca9e692bb00865238 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da63b99fc5f49cca9e692bb00865238, State: Initialized, Role: FOLLOWER
I20260925 07:27:09.385918 17502 raft_consensus.cc:374] T 54bfdbcc0d184e9ea3450858d7c79dfa P 131634a7a8274f75b01beb82e3850c60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:27:09.385963 17502 raft_consensus.cc:729] T 54bfdbcc0d184e9ea3450858d7c79dfa P 131634a7a8274f75b01beb82e3850c60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 131634a7a8274f75b01beb82e3850c60, State: Initialized, Role: FOLLOWER
I20260925 07:27:09.386023 17503 consensus_queue.cc:260] T 54bfdbcc0d184e9ea3450858d7c79dfa P 3da63b99fc5f49cca9e692bb00865238 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "279b2843fb1240e3bec794ac0b04bfb2" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 41809 } } peers { permanent_uuid: "3da63b99fc5f49cca9e692bb00865238" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 40527 } } peers { permanent_uuid: "131634a7a8274f75b01beb82e3850c60" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41407 } }
I20260925 07:27:09.386090 17502 consensus_queue.cc:260] T 54bfdbcc0d184e9ea3450858d7c79dfa P 131634a7a8274f75b01beb82e3850c60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "279b2843fb1240e3bec794ac0b04bfb2" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 41809 } } peers { permanent_uuid: "3da63b99fc5f49cca9e692bb00865238" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 40527 } } peers { permanent_uuid: "131634a7a8274f75b01beb82e3850c60" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41407 } }
I20260925 07:27:09.386257 17503 ts_tablet_manager.cc:1428] T 54bfdbcc0d184e9ea3450858d7c79dfa P 3da63b99fc5f49cca9e692bb00865238: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 07:27:09.386344 17502 ts_tablet_manager.cc:1428] T 54bfdbcc0d184e9ea3450858d7c79dfa P 131634a7a8274f75b01beb82e3850c60: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 07:27:09.386369 17481 heartbeater.cc:503] Master 127.5.158.126:33431 was elected leader, sending a full tablet report...
I20260925 07:27:09.386514 17209 heartbeater.cc:503] Master 127.5.158.126:33431 was elected leader, sending a full tablet report...
W20260925 07:27:09.421284 17210 tablet.cc:2369] T 54bfdbcc0d184e9ea3450858d7c79dfa P 131634a7a8274f75b01beb82e3850c60: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 07:27:09.435530 17506 raft_consensus.cc:482] T 54bfdbcc0d184e9ea3450858d7c79dfa P 3da63b99fc5f49cca9e692bb00865238 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 07:27:09.435669 17506 raft_consensus.cc:504] T 54bfdbcc0d184e9ea3450858d7c79dfa P 3da63b99fc5f49cca9e692bb00865238 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "279b2843fb1240e3bec794ac0b04bfb2" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 41809 } } peers { permanent_uuid: "3da63b99fc5f49cca9e692bb00865238" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 40527 } } peers { permanent_uuid: "131634a7a8274f75b01beb82e3850c60" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41407 } }
I20260925 07:27:09.436054 17506 leader_election.cc:288] T 54bfdbcc0d184e9ea3450858d7c79dfa P 3da63b99fc5f49cca9e692bb00865238 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 279b2843fb1240e3bec794ac0b04bfb2 (127.5.158.66:41809), 131634a7a8274f75b01beb82e3850c60 (127.5.158.65:41407)
I20260925 07:27:09.439929 17483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:09.436201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.66:41809 (local address 127.5.158.67:34933)
0925 07:27:09.436291 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:09.436302 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 07:27:09.436343 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:27:09.436614 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:27:09.436615 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:27:09.436617 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:27:09.436680 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:09.436682 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:09.437571 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:09.437573 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:09.438690 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:09.438691 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:09.439689 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:09.439690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:09.439764 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:09.439768 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 07:27:09.439812 (+ 44us) client_negotiation.cc:240] Negotiation successful
0925 07:27:09.439861 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260925 07:27:09.440191 17496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:09.436419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:34933 (local address 127.5.158.66:41809)
0925 07:27:09.436460 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:09.436462 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:09.436470 (+ 8us) server_negotiation.cc:407] Connection header received
0925 07:27:09.436510 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:09.436511 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:09.436520 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:09.436565 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:27:09.436751 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:09.437460 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:09.439079 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:09.439589 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:09.439652 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:09.439661 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:09.440077 (+ 416us) server_negotiation.cc:299] Negotiation successful
0925 07:27:09.440128 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 07:27:09.440462 17296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54bfdbcc0d184e9ea3450858d7c79dfa" candidate_uuid: "3da63b99fc5f49cca9e692bb00865238" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "279b2843fb1240e3bec794ac0b04bfb2" is_pre_election: true
I20260925 07:27:09.440646 17296 raft_consensus.cc:2454] T 54bfdbcc0d184e9ea3450858d7c79dfa P 279b2843fb1240e3bec794ac0b04bfb2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da63b99fc5f49cca9e692bb00865238 in term 0.
I20260925 07:27:09.440937 17371 leader_election.cc:302] T 54bfdbcc0d184e9ea3450858d7c79dfa P 3da63b99fc5f49cca9e692bb00865238 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 279b2843fb1240e3bec794ac0b04bfb2, 3da63b99fc5f49cca9e692bb00865238; no voters:
I20260925 07:27:09.441021 17498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:09.436430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:55819 (local address 127.5.158.65:41407)
0925 07:27:09.436468 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:09.436470 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:09.437390 (+ 920us) server_negotiation.cc:407] Connection header received
0925 07:27:09.437478 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:09.437479 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:09.437490 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:09.437520 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:27:09.437714 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:09.438394 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:09.439789 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:09.440390 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:09.440441 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:09.440450 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:09.440911 (+ 461us) server_negotiation.cc:299] Negotiation successful
0925 07:27:09.440946 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 07:27:09.441077 17506 raft_consensus.cc:2781] T 54bfdbcc0d184e9ea3450858d7c79dfa P 3da63b99fc5f49cca9e692bb00865238 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 07:27:09.441143 17506 raft_consensus.cc:482] T 54bfdbcc0d184e9ea3450858d7c79dfa P 3da63b99fc5f49cca9e692bb00865238 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 07:27:09.441181 17506 raft_consensus.cc:3037] T 54bfdbcc0d184e9ea3450858d7c79dfa P 3da63b99fc5f49cca9e692bb00865238 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:27:09.441431 17508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:09.436201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:41407 (local address 127.5.158.67:55819)
0925 07:27:09.437328 (+ 1127us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:09.437338 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 07:27:09.437413 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:27:09.437581 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:27:09.437581 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:27:09.437583 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:27:09.437646 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:09.437647 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:09.438524 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:09.438525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:09.439707 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:09.439709 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:09.440755 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:09.440756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:09.440823 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:09.440826 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 07:27:09.441273 (+ 447us) client_negotiation.cc:240] Negotiation successful
0925 07:27:09.441331 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1110,"mutex_wait_us":9,"thread_start_us":91,"threads_started":1}
I20260925 07:27:09.441742 17163 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54bfdbcc0d184e9ea3450858d7c79dfa" candidate_uuid: "3da63b99fc5f49cca9e692bb00865238" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "131634a7a8274f75b01beb82e3850c60" is_pre_election: true
I20260925 07:27:09.441874 17163 raft_consensus.cc:2454] T 54bfdbcc0d184e9ea3450858d7c79dfa P 131634a7a8274f75b01beb82e3850c60 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3da63b99fc5f49cca9e692bb00865238 in term 0.
I20260925 07:27:09.442148 17506 raft_consensus.cc:504] T 54bfdbcc0d184e9ea3450858d7c79dfa P 3da63b99fc5f49cca9e692bb00865238 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "279b2843fb1240e3bec794ac0b04bfb2" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 41809 } } peers { permanent_uuid: "3da63b99fc5f49cca9e692bb00865238" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 40527 } } peers { permanent_uuid: "131634a7a8274f75b01beb82e3850c60" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41407 } }
I20260925 07:27:09.442292 17506 leader_election.cc:288] T 54bfdbcc0d184e9ea3450858d7c79dfa P 3da63b99fc5f49cca9e692bb00865238 [CANDIDATE]: Term 1 election: Requested vote from peers 279b2843fb1240e3bec794ac0b04bfb2 (127.5.158.66:41809), 131634a7a8274f75b01beb82e3850c60 (127.5.158.65:41407)
I20260925 07:27:09.442437 17163 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54bfdbcc0d184e9ea3450858d7c79dfa" candidate_uuid: "3da63b99fc5f49cca9e692bb00865238" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "131634a7a8274f75b01beb82e3850c60"
I20260925 07:27:09.442509 17163 raft_consensus.cc:3037] T 54bfdbcc0d184e9ea3450858d7c79dfa P 131634a7a8274f75b01beb82e3850c60 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:27:09.442513 17296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54bfdbcc0d184e9ea3450858d7c79dfa" candidate_uuid: "3da63b99fc5f49cca9e692bb00865238" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "279b2843fb1240e3bec794ac0b04bfb2"
I20260925 07:27:09.442584 17296 raft_consensus.cc:3037] T 54bfdbcc0d184e9ea3450858d7c79dfa P 279b2843fb1240e3bec794ac0b04bfb2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:27:09.443284 17163 raft_consensus.cc:2454] T 54bfdbcc0d184e9ea3450858d7c79dfa P 131634a7a8274f75b01beb82e3850c60 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da63b99fc5f49cca9e692bb00865238 in term 1.
I20260925 07:27:09.443444 17369 leader_election.cc:302] T 54bfdbcc0d184e9ea3450858d7c79dfa P 3da63b99fc5f49cca9e692bb00865238 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 131634a7a8274f75b01beb82e3850c60, 3da63b99fc5f49cca9e692bb00865238; no voters:
I20260925 07:27:09.443625 17506 raft_consensus.cc:2781] T 54bfdbcc0d184e9ea3450858d7c79dfa P 3da63b99fc5f49cca9e692bb00865238 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:27:09.443784 17506 raft_consensus.cc:686] T 54bfdbcc0d184e9ea3450858d7c79dfa P 3da63b99fc5f49cca9e692bb00865238 [term 1 LEADER]: Becoming Leader. State: Replica: 3da63b99fc5f49cca9e692bb00865238, State: Running, Role: LEADER
I20260925 07:27:09.443861 17506 consensus_queue.cc:237] T 54bfdbcc0d184e9ea3450858d7c79dfa P 3da63b99fc5f49cca9e692bb00865238 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "279b2843fb1240e3bec794ac0b04bfb2" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 41809 } } peers { permanent_uuid: "3da63b99fc5f49cca9e692bb00865238" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 40527 } } peers { permanent_uuid: "131634a7a8274f75b01beb82e3850c60" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41407 } }
I20260925 07:27:09.444001 17296 raft_consensus.cc:2454] T 54bfdbcc0d184e9ea3450858d7c79dfa P 279b2843fb1240e3bec794ac0b04bfb2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3da63b99fc5f49cca9e692bb00865238 in term 1.
I20260925 07:27:09.444653 16995 catalog_manager.cc:5495] T 54bfdbcc0d184e9ea3450858d7c79dfa P 3da63b99fc5f49cca9e692bb00865238 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3da63b99fc5f49cca9e692bb00865238 (127.5.158.67). New cstate: current_term: 1 leader_uuid: "3da63b99fc5f49cca9e692bb00865238" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "279b2843fb1240e3bec794ac0b04bfb2" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 41809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da63b99fc5f49cca9e692bb00865238" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 40527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "131634a7a8274f75b01beb82e3850c60" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41407 } health_report { overall_health: UNKNOWN } } }
I20260925 07:27:09.463066 17495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:09.459722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39484 (local address 127.5.158.67:40527)
0925 07:27:09.459766 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:09.459768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:09.459814 (+ 46us) server_negotiation.cc:407] Connection header received
0925 07:27:09.459852 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:09.459853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:09.459873 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:09.459954 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 07:27:09.460366 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:09.461028 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:09.462093 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:09.462383 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:09.462454 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:09.462658 (+ 204us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 07:27:09.462784 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 07:27:09.462872 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:09.462968 (+ 96us) server_negotiation.cc:299] Negotiation successful
0925 07:27:09.462990 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 07:27:09.464937 17163 raft_consensus.cc:1261] T 54bfdbcc0d184e9ea3450858d7c79dfa P 131634a7a8274f75b01beb82e3850c60 [term 1 FOLLOWER]: Refusing update from remote peer 3da63b99fc5f49cca9e692bb00865238: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:27:09.465317 17509 consensus_queue.cc:1035] T 54bfdbcc0d184e9ea3450858d7c79dfa P 3da63b99fc5f49cca9e692bb00865238 [LEADER]: Connected to new peer: Peer: permanent_uuid: "131634a7a8274f75b01beb82e3850c60" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 41407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:27:09.467198 17296 raft_consensus.cc:1261] T 54bfdbcc0d184e9ea3450858d7c79dfa P 279b2843fb1240e3bec794ac0b04bfb2 [term 1 FOLLOWER]: Refusing update from remote peer 3da63b99fc5f49cca9e692bb00865238: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:27:09.467865 17509 consensus_queue.cc:1035] T 54bfdbcc0d184e9ea3450858d7c79dfa P 3da63b99fc5f49cca9e692bb00865238 [LEADER]: Connected to new peer: Peer: permanent_uuid: "279b2843fb1240e3bec794ac0b04bfb2" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 41809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:27:09.468434 17513 mvcc.cc:205] Tried to move back new op lower bound from 7333155755885658112 to 7333155755802726400. Current Snapshot: MvccSnapshot[applied={T|T < 7333155755885658112}]
I20260925 07:27:09.469925 17512 mvcc.cc:205] Tried to move back new op lower bound from 7333155755885658112 to 7333155755802726400. Current Snapshot: MvccSnapshot[applied={T|T < 7333155755885658112}]
I20260925 07:27:09.472834 16995 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51912:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 07:27:09.472920 16995 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51912:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 07:27:09.473731 16995 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c4cb2ae6904445e2b4201e476d4bfdad: Sending DeleteTablet for 3 replicas of tablet 54bfdbcc0d184e9ea3450858d7c79dfa
I20260925 07:27:09.474210 17281 tablet_service.cc:1514] Processing DeleteTablet for tablet 54bfdbcc0d184e9ea3450858d7c79dfa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:27:09 UTC) from {username='kudu', principal='kudu/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:33840
I20260925 07:27:09.474484 17415 tablet_service.cc:1514] Processing DeleteTablet for tablet 54bfdbcc0d184e9ea3450858d7c79dfa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:27:09 UTC) from {username='kudu', principal='kudu/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:39472
I20260925 07:27:09.474488 17143 tablet_service.cc:1514] Processing DeleteTablet for tablet 54bfdbcc0d184e9ea3450858d7c79dfa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:27:09 UTC) from {username='kudu', principal='kudu/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:56772
I20260925 07:27:09.474876 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 17078
I20260925 07:27:09.474889 17522 tablet_replica.cc:330] T 54bfdbcc0d184e9ea3450858d7c79dfa P 279b2843fb1240e3bec794ac0b04bfb2: stopping tablet replica
I20260925 07:27:09.475466 17522 raft_consensus.cc:2229] T 54bfdbcc0d184e9ea3450858d7c79dfa P 279b2843fb1240e3bec794ac0b04bfb2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 07:27:09.475621 17522 raft_consensus.cc:2258] T 54bfdbcc0d184e9ea3450858d7c79dfa P 279b2843fb1240e3bec794ac0b04bfb2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:27:09.476051 17522 ts_tablet_manager.cc:1905] T 54bfdbcc0d184e9ea3450858d7c79dfa P 279b2843fb1240e3bec794ac0b04bfb2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:27:09.477572 17523 tablet_replica.cc:330] T 54bfdbcc0d184e9ea3450858d7c79dfa P 3da63b99fc5f49cca9e692bb00865238: stopping tablet replica
I20260925 07:27:09.477699 17523 raft_consensus.cc:2229] T 54bfdbcc0d184e9ea3450858d7c79dfa P 3da63b99fc5f49cca9e692bb00865238 [term 1 LEADER]: Raft consensus shutting down.
I20260925 07:27:09.477841 17523 raft_consensus.cc:2258] T 54bfdbcc0d184e9ea3450858d7c79dfa P 3da63b99fc5f49cca9e692bb00865238 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:27:09.478274 17523 ts_tablet_manager.cc:1905] T 54bfdbcc0d184e9ea3450858d7c79dfa P 3da63b99fc5f49cca9e692bb00865238: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:27:09.479669 17523 ts_tablet_manager.cc:1918] T 54bfdbcc0d184e9ea3450858d7c79dfa P 3da63b99fc5f49cca9e692bb00865238: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 07:27:09.479815 17522 ts_tablet_manager.cc:1918] T 54bfdbcc0d184e9ea3450858d7c79dfa P 279b2843fb1240e3bec794ac0b04bfb2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 07:27:09.479916 17522 log.cc:1196] T 54bfdbcc0d184e9ea3450858d7c79dfa P 279b2843fb1240e3bec794ac0b04bfb2: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ts-1/wal/wals/54bfdbcc0d184e9ea3450858d7c79dfa
I20260925 07:27:09.480249 17522 ts_tablet_manager.cc:1939] T 54bfdbcc0d184e9ea3450858d7c79dfa P 279b2843fb1240e3bec794ac0b04bfb2: Deleting consensus metadata
I20260925 07:27:09.480621 17523 log.cc:1196] T 54bfdbcc0d184e9ea3450858d7c79dfa P 3da63b99fc5f49cca9e692bb00865238: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790321110323796-5753-0/minicluster-data/ts-2/wal/wals/54bfdbcc0d184e9ea3450858d7c79dfa
I20260925 07:27:09.480675 16984 catalog_manager.cc:4841] TS 279b2843fb1240e3bec794ac0b04bfb2 (127.5.158.66:41809): tablet 54bfdbcc0d184e9ea3450858d7c79dfa (table test-table [id=68a5abc9b63f4c8fba443cf211f48040]) successfully deleted
I20260925 07:27:09.480955 17523 ts_tablet_manager.cc:1939] T 54bfdbcc0d184e9ea3450858d7c79dfa P 3da63b99fc5f49cca9e692bb00865238: Deleting consensus metadata
I20260925 07:27:09.481336 16981 catalog_manager.cc:4841] TS 3da63b99fc5f49cca9e692bb00865238 (127.5.158.67:40527): tablet 54bfdbcc0d184e9ea3450858d7c79dfa (table test-table [id=68a5abc9b63f4c8fba443cf211f48040]) successfully deleted
W20260925 07:27:09.482415 16982 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.158.65:41407 (error 108)
I20260925 07:27:09.485214 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 17217
I20260925 07:27:09.489348 17497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:09.485716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:41407 (local address 127.0.0.1:56788)
0925 07:27:09.489275 (+ 3559us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:09.489317 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.65:41407: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3537}
W20260925 07:27:09.489462 16982 catalog_manager.cc:4584] TS 131634a7a8274f75b01beb82e3850c60 (127.5.158.65:41407): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 54bfdbcc0d184e9ea3450858d7c79dfa: Network error: Client connection negotiation failed: client connection to 127.5.158.65:41407: connect: Connection refused (error 111)
I20260925 07:27:09.491379 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 17352
I20260925 07:27:09.496129 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 16965
I20260925 07:27:09.500468 5753 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260925 07:27:09.844141 5753 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260925 07:27:09.844238 5753 mini_ranger.cc:66] Stopping Ranger...
I20260925 07:27:10.188032 5753 mini_ranger.cc:68] Stopped Ranger
2026-09-25 07:27:10.188 UTC [16551] LOG: received smart shutdown request
2026-09-25 07:27:10.190 UTC [16551] LOG: background worker "logical replication launcher" (PID 16560) exited with exit code 1
2026-09-25 07:27:10.191 UTC [16555] LOG: shutting down
2026-09-25 07:27:10.333 UTC [16551] LOG: database system is shut down
2026-09-25T07:27:10Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (40968 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 07:27:10 dist-test-slave-1drq krb5kdc[17534](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 07:27:10 dist-test-slave-1drq krb5kdc[17534](info): set up 2 sockets
Sep 25 07:27:10 dist-test-slave-1drq krb5kdc[17534](info): commencing operation
krb5kdc: starting...
W20260925 07:27:11.444937 5753 mini_kdc.cc:121] Time spent starting KDC: real 0.978s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Sep 25 07:27:11 dist-test-slave-1drq krb5kdc[17534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321231, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T07:27:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T07:27:11Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.5.158.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.5.158.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.5.158.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/rangeradmin_127.5.158.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.5.158.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/rangeradmin_127.5.158.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.5.158.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.5.158.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.5.158.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/rangerlookup_127.5.158.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.5.158.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/rangerlookup_127.5.158.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.158.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.5.158.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.158.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/HTTP_127.5.158.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.158.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/HTTP_127.5.158.84@KRBTEST.COM.keytab.
I20260925 07:27:11.595263 5753 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskROew7s/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/postgres -l logfile start
2026-09-25 07:27:13.794 UTC [17564] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-25 07:27:13.794 UTC [17564] LOG: listening on IPv4 address "127.5.158.84", port 45833
2026-09-25 07:27:13.797 UTC [17564] LOG: listening on Unix socket "/tmp/.s.PGSQL.45833"
2026-09-25 07:27:13.815 UTC [17567] LOG: database system was shut down at 2026-09-25 07:27:12 UTC
2026-09-25 07:27:13.818 UTC [17564] LOG: database system is ready to accept connections
I20260925 07:27:14.721796 5753 mini_postgres.cc:96] Postgres bound to 45833
2026-09-25 07:27:14.727 UTC [17575] FATAL: database "slave" does not exist
127.5.158.84:45833 - accepting connections
I20260925 07:27:14.727969 5753 mini_ranger.cc:161] Starting Ranger...
I20260925 07:27:14.741505 5753 mini_ranger.cc:84] Created miniranger Postgres user
I20260925 07:27:15.132768 5753 mini_ranger.cc:87] Created ranger Postgres database
I20260925 07:27:15.132849 5753 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ranger-admin
2026-09-25 07:27:15,387 [I] DB FLAVOR :POSTGRES
2026-09-25 07:27:15,388 [I] --------- Verifying Ranger DB connection ---------
2026-09-25 07:27:15,388 [I] Checking connection..
2026-09-25 07:27:15,388 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskROew7s/build/release/bin/postgresql.jar:/tmp/dist-test-taskROew7s/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.158.84:45833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-25 07:27:15,614 [I] Checking connection passed.
2026-09-25 07:27:15,614 [I] --------- Verifying version history table ---------
2026-09-25 07:27:15,615 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskROew7s/build/release/bin/postgresql.jar:/tmp/dist-test-taskROew7s/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.158.84:45833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 07:27:15,837 [I] Table x_db_version_h does not exist in database ranger
2026-09-25 07:27:15,838 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskROew7s/build/release/bin/postgresql.jar:/tmp/dist-test-taskROew7s/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.158.84:45833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 07:27:16,061 [I] Table x_db_version_h does not exist in database ranger
2026-09-25 07:27:16,062 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-25 07:27:16,062 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskROew7s/build/release/bin/postgresql.jar:/tmp/dist-test-taskROew7s/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.158.84:45833/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskROew7s/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-25 07:27:16.259 UTC [17639] WARNING: there is no transaction in progress
2026-09-25 07:27:16,265 [I] create_dbversion_catalog.sql file imported successfully
2026-09-25 07:27:16,265 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskROew7s/build/release/bin/postgresql.jar:/tmp/dist-test-taskROew7s/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.158.84:45833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 07:27:16,483 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-25 07:27:16,483 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-25 07:27:16,484 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskROew7s/build/release/bin/postgresql.jar:/tmp/dist-test-taskROew7s/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.158.84:45833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-25 07:27:16,681 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskROew7s/build/release/bin/postgresql.jar:/tmp/dist-test-taskROew7s/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.158.84:45833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-25 07:27:16,884 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskROew7s/build/release/bin/postgresql.jar:/tmp/dist-test-taskROew7s/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.158.84:45833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-1drq.c.gcp-upstream.internal','N') ;"
2026-09-25 07:27:17,078 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskROew7s/build/release/bin/postgresql.jar:/tmp/dist-test-taskROew7s/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.158.84:45833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-25 07:27:17,287 [I] Table x_portal_user does not exist in database ranger
2026-09-25 07:27:17,288 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskROew7s/build/release/bin/postgresql.jar:/tmp/dist-test-taskROew7s/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.158.84:45833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-25 07:27:17,491 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-25 07:27:17,491 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskROew7s/build/release/bin/postgresql.jar:/tmp/dist-test-taskROew7s/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.158.84:45833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-25 07:27:17,696 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskROew7s/build/release/bin/postgresql.jar:/tmp/dist-test-taskROew7s/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.158.84:45833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-25 07:27:17,917 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-25 07:27:17,917 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskROew7s/build/release/bin/postgresql.jar:/tmp/dist-test-taskROew7s/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.158.84:45833/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskROew7s/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-25 07:27:18.744 UTC [17747] WARNING: there is no transaction in progress
2026-09-25 07:27:18.757 UTC [17747] WARNING: there is no transaction in progress
2026-09-25 07:27:18.772 UTC [17747] WARNING: there is no transaction in progress
2026-09-25 07:27:18.785 UTC [17747] WARNING: there is no transaction in progress
2026-09-25 07:27:18.800 UTC [17747] WARNING: there is no transaction in progress
2026-09-25 07:27:18.818 UTC [17747] WARNING: there is no transaction in progress
2026-09-25 07:27:18.867 UTC [17747] WARNING: there is no transaction in progress
2026-09-25 07:27:18.873 UTC [17747] WARNING: there is no transaction in progress
2026-09-25 07:27:18.885 UTC [17747] WARNING: there is no transaction in progress
2026-09-25 07:27:18.894 UTC [17747] WARNING: there is no transaction in progress
2026-09-25 07:27:18.903 UTC [17747] WARNING: there is no transaction in progress
2026-09-25 07:27:18.915 UTC [17747] WARNING: there is no transaction in progress
2026-09-25 07:27:18.923 UTC [17747] WARNING: there is no transaction in progress
2026-09-25 07:27:18.928 UTC [17747] WARNING: there is no transaction in progress
2026-09-25 07:27:18.934 UTC [17747] WARNING: there is no transaction in progress
2026-09-25 07:27:18.940 UTC [17747] WARNING: there is no transaction in progress
2026-09-25 07:27:19,492 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-25 07:27:19,492 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskROew7s/build/release/bin/postgresql.jar:/tmp/dist-test-taskROew7s/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.158.84:45833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-25 07:27:19,688 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-25 07:27:19,688 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskROew7s/build/release/bin/postgresql.jar:/tmp/dist-test-taskROew7s/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.158.84:45833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-25 07:27:19,913 [I] Table x_portal_user already exists in database 'ranger'
2026-09-25 07:27:19,913 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskROew7s/build/release/bin/postgresql.jar:/tmp/dist-test-taskROew7s/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.158.84:45833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-25 07:27:20,140 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-25 07:27:20,141 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskROew7s/build/release/bin/postgresql.jar:/tmp/dist-test-taskROew7s/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.158.84:45833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-25 07:27:20,366 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskROew7s/build/release/bin/postgresql.jar:/tmp/dist-test-taskROew7s/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.158.84:45833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-25 07:27:20,597 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskROew7s/build/release/bin/postgresql.jar:/tmp/dist-test-taskROew7s/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.158.84:45833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-1drq.c.gcp-upstream.internal';"
2026-09-25 07:27:20,821 [I] CORE_DB_SCHEMA import status has been updated
2026-09-25 07:27:20,821 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskROew7s/build/release/bin/postgresql.jar:/tmp/dist-test-taskROew7s/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.158.84:45833/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-25 07:27:21,078 [I] DB_PATCHES have already been applied
I20260925 07:27:21.083662 5753 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ranger-admin:/tmp/dist-test-taskROew7s/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskROew7s/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskROew7s/build/release/bin/postgresql.jar:/tmp/dist-test-taskROew7s/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260925 07:27:21.083716 5753 mini_ranger.cc:193] Using host: 127.5.158.84
I20260925 07:27:21.086577 5753 mini_ranger.cc:227] Ranger admin URL: http://127.5.158.84:48619
Sep 25, 2026 7:27:21 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 25, 2026 7:27:21 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 25, 2026 7:27:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ranger-admin/ews/webapp
Sep 25, 2026 7:27:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 25, 2026 7:27:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 25, 2026 7:27:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ranger-admin/ews/webapp].
Sep 25, 2026 7:27:21 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260925 07:27:22.100042 5753 mini_ranger.cc:160] Time spent starting Ranger: real 7.372s user 0.001s sys 0.007s
Sep 25, 2026 7:27:39 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 25, 2026 7:27:39 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 25, 2026 7:27:39 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 25, 2026 7:27:39 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 25, 2026 7:27:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 25, 2026 7:27:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 25, 2026 7:27:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 25, 2026 7:27:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 25, 2026 7:27:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 25, 2026 7:27:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 25, 2026 7:27:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 25, 2026 7:27:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 25, 2026 7:27:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 25, 2026 7:27:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 25, 2026 7:27:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 25, 2026 7:27:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 25, 2026 7:27:39 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 25, 2026 7:27:40 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260925 07:27:40.731845 5753 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.5.158.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.5.158.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.5.158.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/rangerkms_127.5.158.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.5.158.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/rangerkms_127.5.158.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.158.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.5.158.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.158.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/HTTP_127.5.158.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.158.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/HTTP_127.5.158.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 25 07:27:40 dist-test-slave-1drq krb5kdc[17534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321260, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260925 07:27:40.807716 5753 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260925 07:27:40.816555 5753 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260925 07:27:41.540256 5753 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260925 07:27:41.540334 5753 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ranger-kms
I20260925 07:27:41.540373 5753 mini_ranger_kms.cc:227] Using postgres at 127.5.158.84:45833
2026-09-25 07:27:41,691 [I] DB FLAVOR :POSTGRES
2026-09-25 07:27:41,691 [I] --------- Verifying Ranger DB connection ---------
2026-09-25 07:27:41,692 [I] Checking connection
2026-09-25 07:27:41,692 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskROew7s/build/release/bin/postgresql.jar:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.158.84:45833/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-25 07:27:41,909 [I] connection success
2026-09-25 07:27:41,909 [I] --------- Verifying Ranger DB tables ---------
2026-09-25 07:27:41,909 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-25 07:27:41,910 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskROew7s/build/release/bin/postgresql.jar:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.158.84:45833/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-25 07:27:42,129 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-25 07:27:42,129 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-25 07:27:42,129 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-25 07:27:42,130 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskROew7s/build/release/bin/postgresql.jar:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.158.84:45833/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-25 07:27:42,364 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260925 07:27:42.746626 5753 mini_ranger_kms.cc:310] Created kms service
I20260925 07:27:42.934337 5753 mini_ranger_kms.cc:326] Created kudu user
I20260925 07:27:42.972165 5753 mini_ranger_kms.cc:343] Created rangerkms user
Sep 25 07:27:42 dist-test-slave-1drq krb5kdc[17534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321260, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.5.158.84@KRBTEST.COM
I20260925 07:27:43.341539 5753 mini_ranger_kms.cc:384] Added ranger policy
I20260925 07:27:43.341655 5753 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ranger-kms:/tmp/dist-test-taskROew7s/build/release/bin/postgresql.jar:/tmp/dist-test-taskROew7s/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskROew7s/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260925 07:27:43.341671 5753 mini_ranger_kms.cc:240] Using host: 127.5.158.84
I20260925 07:27:43.344878 5753 mini_ranger_kms.cc:276] Ranger KMS PID: 17927
I20260925 07:27:43.344952 5753 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.5.158.84:52617
Sep 25, 2026 7:27:44 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 25, 2026 7:27:44 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 25, 2026 7:27:44 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 25, 2026 7:27:44 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 25, 2026 7:27:44 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ranger-kms/ews/webapp].
Sep 25, 2026 7:27:44 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Sep 25 07:27:45 dist-test-slave-1drq krb5kdc[17534](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790321265, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.158.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:27:45.373231 5753 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 4.566s user 0.000s sys 0.136s
I20260925 07:27:45.373328 5753 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260925 07:27:45.373384 5753 mini_ranger_kms.cc:401] 127.5.158.84:52617/kms/v1/keys
Sep 25 07:27:45 dist-test-slave-1drq krb5kdc[17534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321265, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.158.84@KRBTEST.COM for HTTP/127.5.158.84@KRBTEST.COM
Sep 25 07:27:46 dist-test-slave-1drq krb5kdc[17534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321265, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.158.84@KRBTEST.COM for HTTP/127.5.158.84@KRBTEST.COM
Sep 25 07:27:48 dist-test-slave-1drq krb5kdc[17534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321268, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.5.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
I20260925 07:27:48.079658 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:37921
--webserver_interface=127.5.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:45483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.158.126:37921
--ranger_config_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.158.84:52617/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:27:48.152287 17977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:27:48.152446 17977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:27:48.152482 17977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:27:48.153555 17977 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:27:48.153596 17977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:27:48.153609 17977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:27:48.153622 17977 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:27:48.153671 17977 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:27:48.155149 17977 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:45483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/master-0/wal
--ranger_kms_url=127.5.158.84:52617/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.5.158.126:37921
--ranger_config_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:37921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17977
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:27:48.155296 17977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:27:48.155503 17977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 07:27:48.159948 17977 server_base.cc:1034] running on GCE node
W20260925 07:27:48.159915 17985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:27:48.159927 17983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:27:48.160177 17982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:27:48.160404 17977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:27:48.160640 17977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:27:48.161803 17977 hybrid_clock.cc:648] HybridClock initialized: now 1790321268161794 us; error 24 us; skew 500 ppm
Sep 25 07:27:48 dist-test-slave-1drq krb5kdc[17534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321268, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:27:48.167413 17977 init.cc:377] Logged in from keytab as kudu/127.5.158.126@KRBTEST.COM (short username kudu)
I20260925 07:27:48.167773 17977 webserver.cc:460] Webserver started at http://127.5.158.126:44043/ using document root <none> and password file <none>
I20260925 07:27:48.167956 17977 fs_manager.cc:359] Metadata directory not provided
I20260925 07:27:48.167990 17977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:27:48.168099 17977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 07:27:48 dist-test-slave-1drq krb5kdc[17534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321268, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.126@KRBTEST.COM for HTTP/127.5.158.84@KRBTEST.COM
I20260925 07:27:48.215097 17977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/master-0/data/instance:
uuid: "bbb0f321adbd492ab6df0b3d69b8bc9d"
format_stamp: "Formatted at 2026-09-25 07:27:48 on dist-test-slave-1drq"
server_key: "ea5cb01594bf70dc93398c03be69b8cc"
server_key_iv: "0e00be4a6faa808280de870d9cdc5cf3"
server_key_version: "kuduclusterkey@0"
I20260925 07:27:48.215507 17977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/master-0/wal/instance:
uuid: "bbb0f321adbd492ab6df0b3d69b8bc9d"
format_stamp: "Formatted at 2026-09-25 07:27:48 on dist-test-slave-1drq"
server_key: "ea5cb01594bf70dc93398c03be69b8cc"
server_key_iv: "0e00be4a6faa808280de870d9cdc5cf3"
server_key_version: "kuduclusterkey@0"
I20260925 07:27:48.217140 17977 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 07:27:48.226616 17992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:48.226781 17977 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:27:48.226857 17977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "bbb0f321adbd492ab6df0b3d69b8bc9d"
format_stamp: "Formatted at 2026-09-25 07:27:48 on dist-test-slave-1drq"
server_key: "ea5cb01594bf70dc93398c03be69b8cc"
server_key_iv: "0e00be4a6faa808280de870d9cdc5cf3"
server_key_version: "kuduclusterkey@0"
I20260925 07:27:48.226940 17977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:27:48.256376 17977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:27:48.258141 17977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:27:48.258296 17977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:27:48.262599 17977 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.126:37921
I20260925 07:27:48.262670 18044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.126:37921 every 8 connection(s)
I20260925 07:27:48.262977 17977 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:27:48.263677 18045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:27:48.265720 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 17977
I20260925 07:27:48.265827 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/master-0/wal/instance
I20260925 07:27:48.266001 5753 external_mini_cluster.cc:1428] Setting key c0769a3fbe955af6b913a629944392e6
I20260925 07:27:48.266651 18045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbb0f321adbd492ab6df0b3d69b8bc9d: Bootstrap starting.
I20260925 07:27:48.267351 18045 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bbb0f321adbd492ab6df0b3d69b8bc9d: Neither blocks nor log segments found. Creating new log.
I20260925 07:27:48.267688 18045 log.cc:824] T 00000000000000000000000000000000 P bbb0f321adbd492ab6df0b3d69b8bc9d: Log is configured to *not* fsync() on all Append() calls
I20260925 07:27:48.268308 18045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbb0f321adbd492ab6df0b3d69b8bc9d: No bootstrap required, opened a new log
I20260925 07:27:48.270334 18045 raft_consensus.cc:348] T 00000000000000000000000000000000 P bbb0f321adbd492ab6df0b3d69b8bc9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbb0f321adbd492ab6df0b3d69b8bc9d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37921 } }
I20260925 07:27:48.270505 18045 raft_consensus.cc:374] T 00000000000000000000000000000000 P bbb0f321adbd492ab6df0b3d69b8bc9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:27:48.270560 18045 raft_consensus.cc:729] T 00000000000000000000000000000000 P bbb0f321adbd492ab6df0b3d69b8bc9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbb0f321adbd492ab6df0b3d69b8bc9d, State: Initialized, Role: FOLLOWER
I20260925 07:27:48.270679 18045 consensus_queue.cc:260] T 00000000000000000000000000000000 P bbb0f321adbd492ab6df0b3d69b8bc9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbb0f321adbd492ab6df0b3d69b8bc9d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37921 } }
I20260925 07:27:48.270764 18045 raft_consensus.cc:388] T 00000000000000000000000000000000 P bbb0f321adbd492ab6df0b3d69b8bc9d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 07:27:48.270810 18045 raft_consensus.cc:482] T 00000000000000000000000000000000 P bbb0f321adbd492ab6df0b3d69b8bc9d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 07:27:48.270857 18045 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bbb0f321adbd492ab6df0b3d69b8bc9d [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:27:48.271486 18045 raft_consensus.cc:504] T 00000000000000000000000000000000 P bbb0f321adbd492ab6df0b3d69b8bc9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbb0f321adbd492ab6df0b3d69b8bc9d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37921 } }
I20260925 07:27:48.271582 18045 leader_election.cc:302] T 00000000000000000000000000000000 P bbb0f321adbd492ab6df0b3d69b8bc9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bbb0f321adbd492ab6df0b3d69b8bc9d; no voters:
I20260925 07:27:48.271780 18045 leader_election.cc:288] T 00000000000000000000000000000000 P bbb0f321adbd492ab6df0b3d69b8bc9d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 07:27:48.271826 18050 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bbb0f321adbd492ab6df0b3d69b8bc9d [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:27:48.272058 18050 raft_consensus.cc:686] T 00000000000000000000000000000000 P bbb0f321adbd492ab6df0b3d69b8bc9d [term 1 LEADER]: Becoming Leader. State: Replica: bbb0f321adbd492ab6df0b3d69b8bc9d, State: Running, Role: LEADER
I20260925 07:27:48.272101 18045 sys_catalog.cc:564] T 00000000000000000000000000000000 P bbb0f321adbd492ab6df0b3d69b8bc9d [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:27:48.272219 18050 consensus_queue.cc:237] T 00000000000000000000000000000000 P bbb0f321adbd492ab6df0b3d69b8bc9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbb0f321adbd492ab6df0b3d69b8bc9d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37921 } }
I20260925 07:27:48.272490 18050 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbb0f321adbd492ab6df0b3d69b8bc9d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bbb0f321adbd492ab6df0b3d69b8bc9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbb0f321adbd492ab6df0b3d69b8bc9d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37921 } } }
I20260925 07:27:48.272557 18050 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbb0f321adbd492ab6df0b3d69b8bc9d [sys.catalog]: This master's current role is: LEADER
I20260925 07:27:48.272671 18050 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbb0f321adbd492ab6df0b3d69b8bc9d [sys.catalog]: SysCatalogTable state changed. Reason: New leader bbb0f321adbd492ab6df0b3d69b8bc9d. Latest consensus state: current_term: 1 leader_uuid: "bbb0f321adbd492ab6df0b3d69b8bc9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbb0f321adbd492ab6df0b3d69b8bc9d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37921 } } }
I20260925 07:27:48.272734 18050 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbb0f321adbd492ab6df0b3d69b8bc9d [sys.catalog]: This master's current role is: LEADER
I20260925 07:27:48.272851 18055 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Sep 25 07:27:48 dist-test-slave-1drq krb5kdc[17534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321268, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:27:48.273335 18055 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 07:27:48.273649 18045 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260925 07:27:48.279775 18048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:48.266567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47658 (local address 127.5.158.126:37921)
0925 07:27:48.266706 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:48.266714 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:48.266741 (+ 27us) server_negotiation.cc:407] Connection header received
0925 07:27:48.267130 (+ 389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:48.267135 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:48.267213 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:48.267340 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:27:48.267994 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:48.269384 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:48.270394 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:48.270931 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:48.271043 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:48.273534 (+ 2491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:27:48.273537 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:27:48.273538 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:27:48.273548 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:27:48.277696 (+ 4148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:48.278141 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:48.278143 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:48.278144 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:48.278257 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:48.278472 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:48.278474 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:48.278474 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:48.278807 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:27:48.278941 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:48.279414 (+ 473us) server_negotiation.cc:299] Negotiation successful
0925 07:27:48.279545 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":76,"threads_started":1}
I20260925 07:27:48.281200 18055 catalog_manager.cc:1347] Generated new cluster ID: 333a3ff8d69247f3b5e0bc37a4997c55
I20260925 07:27:48.281252 18055 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 07:27:48.301183 18055 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 07:27:48.301787 18055 catalog_manager.cc:1504] Loading token signing keys...
I20260925 07:27:48.311028 18055 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bbb0f321adbd492ab6df0b3d69b8bc9d: Generated new TSK 0
I20260925 07:27:48.311160 18055 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.158.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab.
Entry for principal kudu/127.5.158.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab.
I20260925 07:27:48.923717 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.158.84:52617/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.65:0
--local_ip_for_outbound_sockets=127.5.158.65
--webserver_interface=127.5.158.65
--webserver_port=0
--tserver_master_addrs=127.5.158.126:37921
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab
--principal=kudu/127.5.158.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:45483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:27:49.020704 18091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:27:49.020865 18091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:27:49.020884 18091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:27:49.021994 18091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:27:49.022033 18091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:27:49.022075 18091 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.65
I20260925 07:27:49.023439 18091 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:45483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.5.158.84:52617/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab
--principal=kudu/127.5.158.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.158.65
--webserver_port=0
--tserver_master_addrs=127.5.158.126:37921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18091
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.65
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:27:49.023627 18091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:27:49.023834 18091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:27:49.029465 18097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:27:49.029479 18096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:27:49.029644 18091 server_base.cc:1034] running on GCE node
W20260925 07:27:49.029675 18099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:27:49.029968 18091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:27:49.030215 18091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:27:49.031371 18091 hybrid_clock.cc:648] HybridClock initialized: now 1790321269031355 us; error 32 us; skew 500 ppm
Sep 25 07:27:49 dist-test-slave-1drq krb5kdc[17534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321269, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:27:49.033752 18091 init.cc:377] Logged in from keytab as kudu/127.5.158.65@KRBTEST.COM (short username kudu)
I20260925 07:27:49.034238 18091 webserver.cc:460] Webserver started at http://127.5.158.65:37647/ using document root <none> and password file <none>
I20260925 07:27:49.034482 18091 fs_manager.cc:359] Metadata directory not provided
I20260925 07:27:49.034556 18091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:27:49.034693 18091 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 07:27:49 dist-test-slave-1drq krb5kdc[17534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321269, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.65@KRBTEST.COM for HTTP/127.5.158.84@KRBTEST.COM
I20260925 07:27:49.046813 18091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-0/data/instance:
uuid: "d36767c07fe04ceab87767a829be1a28"
format_stamp: "Formatted at 2026-09-25 07:27:49 on dist-test-slave-1drq"
server_key: "a5e48257b6025b55ae778a8fc9e4cda9"
server_key_iv: "93eda88cfe4728d91acb419bcf146717"
server_key_version: "kuduclusterkey@0"
I20260925 07:27:49.047260 18091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-0/wal/instance:
uuid: "d36767c07fe04ceab87767a829be1a28"
format_stamp: "Formatted at 2026-09-25 07:27:49 on dist-test-slave-1drq"
server_key: "a5e48257b6025b55ae778a8fc9e4cda9"
server_key_iv: "93eda88cfe4728d91acb419bcf146717"
server_key_version: "kuduclusterkey@0"
I20260925 07:27:49.049280 18091 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260925 07:27:49.059463 18108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:49.059623 18091 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260925 07:27:49.059728 18091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-0/wal
uuid: "d36767c07fe04ceab87767a829be1a28"
format_stamp: "Formatted at 2026-09-25 07:27:49 on dist-test-slave-1drq"
server_key: "a5e48257b6025b55ae778a8fc9e4cda9"
server_key_iv: "93eda88cfe4728d91acb419bcf146717"
server_key_version: "kuduclusterkey@0"
I20260925 07:27:49.059813 18091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:27:49.082444 18091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:27:49.086820 18091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:27:49.086983 18091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:27:49.087208 18091 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:27:49.087558 18091 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:27:49.087607 18091 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:49.087667 18091 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:27:49.087709 18091 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:49.094807 18091 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.65:44029
I20260925 07:27:49.094908 18221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.65:44029 every 8 connection(s)
I20260925 07:27:49.095216 18091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
I20260925 07:27:49.099005 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 18091
I20260925 07:27:49.099097 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-0/wal/instance
I20260925 07:27:49.099272 5753 external_mini_cluster.cc:1428] Setting key 8fcea87d9c28717f845da0a5e3cee783
Sep 25 07:27:49 dist-test-slave-1drq krb5kdc[17534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321269, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.65@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:27:49.104708 18224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:49.096017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:48079 (local address 127.5.158.126:37921)
0925 07:27:49.096141 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:49.096144 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:49.096322 (+ 178us) server_negotiation.cc:407] Connection header received
0925 07:27:49.096812 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:49.096814 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:49.096828 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:49.096887 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:27:49.097468 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:49.097938 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:49.098840 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:49.099127 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:49.099195 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:49.101757 (+ 2562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:27:49.101759 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:27:49.101759 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:27:49.101768 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:27:49.103325 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:49.103718 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:49.103719 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:49.103719 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:49.103785 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:49.104076 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:49.104076 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:49.104077 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:49.104192 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:27:49.104262 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:49.104530 (+ 268us) server_negotiation.cc:299] Negotiation successful
0925 07:27:49.104617 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":71,"threads_started":1}
I20260925 07:27:49.104872 18225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:49.096008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37921 (local address 127.5.158.65:48079)
0925 07:27:49.096198 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:49.096233 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 07:27:49.096709 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:27:49.096958 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:27:49.096962 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:27:49.097195 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:27:49.097373 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:49.097377 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:49.098025 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:49.098026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:49.098746 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:49.098748 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:49.099239 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:49.099240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:49.099383 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:49.099945 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:27:49.099954 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:27:49.101631 (+ 1677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:27:49.103421 (+ 1790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:27:49.103428 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:27:49.103430 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:27:49.103626 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:27:49.103884 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:27:49.103885 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:27:49.103885 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:27:49.104001 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:27:49.104265 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:27:49.104271 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:27:49.104401 (+ 130us) client_negotiation.cc:769] Sending connection context
0925 07:27:49.104510 (+ 109us) client_negotiation.cc:240] Negotiation successful
0925 07:27:49.104644 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":73,"threads_started":1}
I20260925 07:27:49.105489 18222 heartbeater.cc:347] Connected to a master server at 127.5.158.126:37921
I20260925 07:27:49.105573 18222 heartbeater.cc:464] Registering TS with master...
I20260925 07:27:49.105719 18222 heartbeater.cc:511] Master 127.5.158.126:37921 requested a full tablet report, sending...
I20260925 07:27:49.106128 18009 ts_manager.cc:194] Registered new tserver with Master: d36767c07fe04ceab87767a829be1a28 (127.5.158.65:44029)
I20260925 07:27:49.106698 18009 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.65@KRBTEST.COM'} at 127.5.158.65:48079
WARNING: no policy specified for kudu/127.5.158.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab.
Entry for principal kudu/127.5.158.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab.
I20260925 07:27:49.131174 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.158.84:52617/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.66:0
--local_ip_for_outbound_sockets=127.5.158.66
--webserver_interface=127.5.158.66
--webserver_port=0
--tserver_master_addrs=127.5.158.126:37921
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab
--principal=kudu/127.5.158.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:45483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:27:49.217299 18228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:27:49.217494 18228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:27:49.217531 18228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:27:49.219200 18228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:27:49.219267 18228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:27:49.219310 18228 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.66
I20260925 07:27:49.220674 18228 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:45483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.5.158.84:52617/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab
--principal=kudu/127.5.158.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.158.66
--webserver_port=0
--tserver_master_addrs=127.5.158.126:37921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18228
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.66
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:27:49.220890 18228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:27:49.221205 18228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:27:49.228677 18235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:27:49.228638 18238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:27:49.228895 18236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:27:49.229569 18228 server_base.cc:1034] running on GCE node
I20260925 07:27:49.229753 18228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:27:49.229971 18228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
Sep 25 07:27:49 dist-test-slave-1drq krb5kdc[17534](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790321269, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:27:49.237319 18228 hybrid_clock.cc:648] HybridClock initialized: now 1790321269237299 us; error 30 us; skew 500 ppm
Sep 25 07:27:49 dist-test-slave-1drq krb5kdc[17534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321269, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:27:49.240326 18228 init.cc:377] Logged in from keytab as kudu/127.5.158.66@KRBTEST.COM (short username kudu)
I20260925 07:27:49.240783 18228 webserver.cc:460] Webserver started at http://127.5.158.66:41413/ using document root <none> and password file <none>
I20260925 07:27:49.240993 18228 fs_manager.cc:359] Metadata directory not provided
I20260925 07:27:49.241039 18228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:27:49.241181 18228 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 07:27:49 dist-test-slave-1drq krb5kdc[17534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321269, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.66@KRBTEST.COM for HTTP/127.5.158.84@KRBTEST.COM
I20260925 07:27:49.254570 18228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-1/data/instance:
uuid: "2f41581887534ad29bb86fbeee06df42"
format_stamp: "Formatted at 2026-09-25 07:27:49 on dist-test-slave-1drq"
server_key: "cb47d2d927588e0062a72f3c1a7f5427"
server_key_iv: "66381b457f976276c4ad717ffe7739a6"
server_key_version: "kuduclusterkey@0"
I20260925 07:27:49.255064 18228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-1/wal/instance:
uuid: "2f41581887534ad29bb86fbeee06df42"
format_stamp: "Formatted at 2026-09-25 07:27:49 on dist-test-slave-1drq"
server_key: "cb47d2d927588e0062a72f3c1a7f5427"
server_key_iv: "66381b457f976276c4ad717ffe7739a6"
server_key_version: "kuduclusterkey@0"
I20260925 07:27:49.257550 18228 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260925 07:27:49.274191 18245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:49.279089 18228 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.001s sys 0.000s
I20260925 07:27:49.279206 18228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-1/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-1/wal
uuid: "2f41581887534ad29bb86fbeee06df42"
format_stamp: "Formatted at 2026-09-25 07:27:49 on dist-test-slave-1drq"
server_key: "cb47d2d927588e0062a72f3c1a7f5427"
server_key_iv: "66381b457f976276c4ad717ffe7739a6"
server_key_version: "kuduclusterkey@0"
I20260925 07:27:49.279304 18228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:27:49.294121 18228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:27:49.296304 18228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:27:49.296422 18228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:27:49.296595 18228 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:27:49.296896 18228 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:27:49.296943 18228 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:49.296970 18228 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:27:49.296986 18228 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:49.304318 18228 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.66:38887
I20260925 07:27:49.304404 18359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.66:38887 every 8 connection(s)
I20260925 07:27:49.304787 18228 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
I20260925 07:27:49.305570 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 18228
I20260925 07:27:49.305670 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-1/wal/instance
I20260925 07:27:49.305881 5753 external_mini_cluster.cc:1428] Setting key e16df8f30d72a42a488d051630557e0d
Sep 25 07:27:49 dist-test-slave-1drq krb5kdc[17534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321269, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.66@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:27:49.315376 18362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:49.305662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37921 (local address 127.5.158.66:49343)
0925 07:27:49.306167 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:49.306211 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 07:27:49.306782 (+ 571us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:27:49.307021 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:27:49.307025 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:27:49.307193 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:27:49.307365 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:49.307369 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:49.308039 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:49.308040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:49.309262 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:49.309265 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:49.309842 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:49.309843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:49.309974 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:49.310461 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:27:49.310471 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:27:49.312035 (+ 1564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:27:49.313890 (+ 1855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:27:49.313895 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:27:49.313897 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:27:49.314162 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:27:49.314369 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:27:49.314370 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:27:49.314370 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:27:49.314487 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:27:49.314768 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:27:49.314773 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:27:49.314913 (+ 140us) client_negotiation.cc:769] Sending connection context
0925 07:27:49.315018 (+ 105us) client_negotiation.cc:240] Negotiation successful
0925 07:27:49.315161 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":73,"threads_started":1}
I20260925 07:27:49.315923 18224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:49.305958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:49343 (local address 127.5.158.126:37921)
0925 07:27:49.306025 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:49.306028 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:49.306275 (+ 247us) server_negotiation.cc:407] Connection header received
0925 07:27:49.306878 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:49.306879 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:49.306891 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:49.306944 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:27:49.307470 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:49.307935 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:49.309366 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:49.309702 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:49.309770 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:49.312132 (+ 2362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:27:49.312133 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:27:49.312133 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:27:49.312140 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:27:49.313785 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:49.314243 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:49.314244 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:49.314244 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:49.314300 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:49.314567 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:49.314568 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:49.314568 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:49.314688 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:27:49.314744 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:49.315760 (+ 1016us) server_negotiation.cc:299] Negotiation successful
0925 07:27:49.315841 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 07:27:49.316562 18360 heartbeater.cc:347] Connected to a master server at 127.5.158.126:37921
I20260925 07:27:49.316709 18360 heartbeater.cc:464] Registering TS with master...
I20260925 07:27:49.316876 18360 heartbeater.cc:511] Master 127.5.158.126:37921 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.5.158.67@KRBTEST.COM; defaulting to no policy
I20260925 07:27:49.317257 18009 ts_manager.cc:194] Registered new tserver with Master: 2f41581887534ad29bb86fbeee06df42 (127.5.158.66:38887)
I20260925 07:27:49.317718 18009 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.66@KRBTEST.COM'} at 127.5.158.66:49343
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab.
Entry for principal kudu/127.5.158.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab.
I20260925 07:27:49.334447 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.158.84:52617/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.67:0
--local_ip_for_outbound_sockets=127.5.158.67
--webserver_interface=127.5.158.67
--webserver_port=0
--tserver_master_addrs=127.5.158.126:37921
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab
--principal=kudu/127.5.158.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:45483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:27:49.435942 18365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:27:49.436210 18365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:27:49.436256 18365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:27:49.437762 18365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:27:49.437834 18365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:27:49.437882 18365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.67
I20260925 07:27:49.439745 18365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:45483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.5.158.84:52617/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab
--principal=kudu/127.5.158.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.158.67
--webserver_port=0
--tserver_master_addrs=127.5.158.126:37921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.67
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:27:49.439954 18365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:27:49.440189 18365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:27:49.445926 18371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:27:49.445940 18370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:27:49.446245 18373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:27:49.446853 18365 server_base.cc:1034] running on GCE node
I20260925 07:27:49.447042 18365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:27:49.447269 18365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:27:49.448446 18365 hybrid_clock.cc:648] HybridClock initialized: now 1790321269448430 us; error 34 us; skew 500 ppm
Sep 25 07:27:49 dist-test-slave-1drq krb5kdc[17534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321269, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:27:49.451511 18365 init.cc:377] Logged in from keytab as kudu/127.5.158.67@KRBTEST.COM (short username kudu)
I20260925 07:27:49.451984 18365 webserver.cc:460] Webserver started at http://127.5.158.67:45065/ using document root <none> and password file <none>
I20260925 07:27:49.452255 18365 fs_manager.cc:359] Metadata directory not provided
I20260925 07:27:49.452313 18365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:27:49.452430 18365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 07:27:49 dist-test-slave-1drq krb5kdc[17534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321269, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.67@KRBTEST.COM for HTTP/127.5.158.84@KRBTEST.COM
I20260925 07:27:49.468077 18365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-2/data/instance:
uuid: "8c9a642ca63546dda148f11f2974bf01"
format_stamp: "Formatted at 2026-09-25 07:27:49 on dist-test-slave-1drq"
server_key: "2389e88be6adcb082b199a62c31ea4d1"
server_key_iv: "bfe2856ea2e318a39719ebb95e02429e"
server_key_version: "kuduclusterkey@0"
I20260925 07:27:49.468443 18365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-2/wal/instance:
uuid: "8c9a642ca63546dda148f11f2974bf01"
format_stamp: "Formatted at 2026-09-25 07:27:49 on dist-test-slave-1drq"
server_key: "2389e88be6adcb082b199a62c31ea4d1"
server_key_iv: "bfe2856ea2e318a39719ebb95e02429e"
server_key_version: "kuduclusterkey@0"
I20260925 07:27:49.469848 18365 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 07:27:49.483914 18380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:49.484160 18365 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 07:27:49.484273 18365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-2/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-2/wal
uuid: "8c9a642ca63546dda148f11f2974bf01"
format_stamp: "Formatted at 2026-09-25 07:27:49 on dist-test-slave-1drq"
server_key: "2389e88be6adcb082b199a62c31ea4d1"
server_key_iv: "bfe2856ea2e318a39719ebb95e02429e"
server_key_version: "kuduclusterkey@0"
I20260925 07:27:49.484372 18365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:27:49.498613 18365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:27:49.500941 18365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:27:49.501142 18365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:27:49.501403 18365 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:27:49.501816 18365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:27:49.501900 18365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:49.501972 18365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:27:49.502027 18365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:49.510623 18365 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.67:43091
I20260925 07:27:49.510746 18493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.67:43091 every 8 connection(s)
I20260925 07:27:49.511070 18365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
Sep 25 07:27:49 dist-test-slave-1drq krb5kdc[17534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321269, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.67@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:27:49.519764 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 18365
I20260925 07:27:49.519901 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-2/wal/instance
I20260925 07:27:49.520087 5753 external_mini_cluster.cc:1428] Setting key 09a3c2a1cc87e1220133b048e9348efb
I20260925 07:27:49.524897 18496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:49.511882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37921 (local address 127.5.158.67:56725)
0925 07:27:49.512211 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:49.512250 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 07:27:49.512917 (+ 667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:27:49.513285 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:27:49.513290 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:27:49.513504 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:27:49.513734 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:49.513739 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:49.514774 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:49.514775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:49.515989 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:49.515992 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:49.516507 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:49.516509 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:49.516633 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:49.518025 (+ 1392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:27:49.518034 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:27:49.521184 (+ 3150us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:27:49.523177 (+ 1993us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:27:49.523183 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:27:49.523185 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:27:49.523476 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:27:49.523711 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:27:49.523711 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:27:49.523712 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:27:49.523880 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:27:49.524221 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:27:49.524227 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:27:49.524381 (+ 154us) client_negotiation.cc:769] Sending connection context
0925 07:27:49.524498 (+ 117us) client_negotiation.cc:240] Negotiation successful
0925 07:27:49.524659 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":81,"threads_started":1}
I20260925 07:27:49.525421 18224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:49.511749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:56725 (local address 127.5.158.126:37921)
0925 07:27:49.511799 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:49.511801 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:49.512334 (+ 533us) server_negotiation.cc:407] Connection header received
0925 07:27:49.513060 (+ 726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:49.513062 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:49.513073 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:49.513162 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:27:49.513873 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:49.514664 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:49.516089 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:49.516385 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:49.516480 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:49.521326 (+ 4846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:27:49.521328 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:27:49.521328 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:27:49.521336 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:27:49.523081 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:49.523571 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:49.523572 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:49.523573 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:49.523637 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:49.523970 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:49.523971 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:49.523971 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:49.524117 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:27:49.524190 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:49.525240 (+ 1050us) server_negotiation.cc:299] Negotiation successful
0925 07:27:49.525327 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 07:27:49.526230 18494 heartbeater.cc:347] Connected to a master server at 127.5.158.126:37921
I20260925 07:27:49.526329 18494 heartbeater.cc:464] Registering TS with master...
I20260925 07:27:49.526527 18494 heartbeater.cc:511] Master 127.5.158.126:37921 requested a full tablet report, sending...
I20260925 07:27:49.526963 18009 ts_manager.cc:194] Registered new tserver with Master: 8c9a642ca63546dda148f11f2974bf01 (127.5.158.67:43091)
I20260925 07:27:49.527428 18009 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.67@KRBTEST.COM'} at 127.5.158.67:56725
I20260925 07:27:49.531834 5753 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 07:27:49.532588 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 18091
I20260925 07:27:49.541869 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 18228
I20260925 07:27:49.546130 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 18365
I20260925 07:27:49.552238 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 17977
I20260925 07:27:49.557945 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:37921
--webserver_interface=127.5.158.126
--webserver_port=44043
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:45483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.158.126:37921
--ranger_config_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.158.84:52617/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.158.84:52617/kms with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:27:49.629853 18497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:27:49.630005 18497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:27:49.630023 18497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:27:49.631011 18497 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:27:49.631047 18497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:27:49.631059 18497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:27:49.631104 18497 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:27:49.631129 18497 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:27:49.632405 18497 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:45483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.158.84:52617/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.5.158.126:37921
--ranger_config_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:37921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.158.126
--webserver_port=44043
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18497
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:27:49.632596 18497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:27:49.632754 18497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:27:49.636979 18502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:27:49.637009 18503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:27:49.636981 18505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:27:49.637018 18497 server_base.cc:1034] running on GCE node
I20260925 07:27:49.637387 18497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:27:49.637625 18497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:27:49.638769 18497 hybrid_clock.cc:648] HybridClock initialized: now 1790321269638760 us; error 35 us; skew 500 ppm
Sep 25 07:27:49 dist-test-slave-1drq krb5kdc[17534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321269, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:27:49.640787 18497 init.cc:377] Logged in from keytab as kudu/127.5.158.126@KRBTEST.COM (short username kudu)
I20260925 07:27:49.641216 18497 webserver.cc:460] Webserver started at http://127.5.158.126:44043/ using document root <none> and password file <none>
I20260925 07:27:49.641402 18497 fs_manager.cc:359] Metadata directory not provided
I20260925 07:27:49.641463 18497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 25 07:27:49 dist-test-slave-1drq krb5kdc[17534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321269, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.126@KRBTEST.COM for HTTP/127.5.158.84@KRBTEST.COM
I20260925 07:27:49.706220 18497 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 07:27:49.707087 18513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:49.707309 18497 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 07:27:49.707410 18497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "bbb0f321adbd492ab6df0b3d69b8bc9d"
format_stamp: "Formatted at 2026-09-25 07:27:48 on dist-test-slave-1drq"
server_key: "ea5cb01594bf70dc93398c03be69b8cc"
server_key_iv: "0e00be4a6faa808280de870d9cdc5cf3"
server_key_version: "kuduclusterkey@0"
I20260925 07:27:49.707885 18497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:27:49.739497 18497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:27:49.741276 18497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:27:49.741406 18497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:27:49.746440 18497 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.126:37921
I20260925 07:27:49.746461 18565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.126:37921 every 8 connection(s)
I20260925 07:27:49.746870 18497 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:27:49.747910 18566 sys_catalog.cc:263] Verifying existing consensus state
I20260925 07:27:49.748467 18566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbb0f321adbd492ab6df0b3d69b8bc9d: Bootstrap starting.
I20260925 07:27:49.753695 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 18497
I20260925 07:27:49.754055 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.158.84:52617/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.65:44029
--local_ip_for_outbound_sockets=127.5.158.65
--tserver_master_addrs=127.5.158.126:37921
--webserver_port=37647
--webserver_interface=127.5.158.65
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab
--principal=kudu/127.5.158.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:45483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.158.84:52617/kms with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 07:27:49.756055 18566 log.cc:824] T 00000000000000000000000000000000 P bbb0f321adbd492ab6df0b3d69b8bc9d: Log is configured to *not* fsync() on all Append() calls
I20260925 07:27:49.757584 18566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbb0f321adbd492ab6df0b3d69b8bc9d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 07:27:49.757859 18566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbb0f321adbd492ab6df0b3d69b8bc9d: Bootstrap complete.
I20260925 07:27:49.759938 18566 raft_consensus.cc:348] T 00000000000000000000000000000000 P bbb0f321adbd492ab6df0b3d69b8bc9d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbb0f321adbd492ab6df0b3d69b8bc9d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37921 } }
I20260925 07:27:49.760182 18566 raft_consensus.cc:729] T 00000000000000000000000000000000 P bbb0f321adbd492ab6df0b3d69b8bc9d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbb0f321adbd492ab6df0b3d69b8bc9d, State: Initialized, Role: FOLLOWER
I20260925 07:27:49.760339 18566 consensus_queue.cc:260] T 00000000000000000000000000000000 P bbb0f321adbd492ab6df0b3d69b8bc9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbb0f321adbd492ab6df0b3d69b8bc9d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37921 } }
I20260925 07:27:49.760409 18566 raft_consensus.cc:388] T 00000000000000000000000000000000 P bbb0f321adbd492ab6df0b3d69b8bc9d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 07:27:49.760461 18566 raft_consensus.cc:482] T 00000000000000000000000000000000 P bbb0f321adbd492ab6df0b3d69b8bc9d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 07:27:49.760509 18566 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bbb0f321adbd492ab6df0b3d69b8bc9d [term 1 FOLLOWER]: Advancing to term 2
I20260925 07:27:49.761410 18566 raft_consensus.cc:504] T 00000000000000000000000000000000 P bbb0f321adbd492ab6df0b3d69b8bc9d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbb0f321adbd492ab6df0b3d69b8bc9d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37921 } }
I20260925 07:27:49.761509 18566 leader_election.cc:302] T 00000000000000000000000000000000 P bbb0f321adbd492ab6df0b3d69b8bc9d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bbb0f321adbd492ab6df0b3d69b8bc9d; no voters:
I20260925 07:27:49.761742 18566 leader_election.cc:288] T 00000000000000000000000000000000 P bbb0f321adbd492ab6df0b3d69b8bc9d [CANDIDATE]: Term 2 election: Requested vote from peers
I20260925 07:27:49.761771 18570 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bbb0f321adbd492ab6df0b3d69b8bc9d [term 2 FOLLOWER]: Leader election won for term 2
I20260925 07:27:49.761960 18570 raft_consensus.cc:686] T 00000000000000000000000000000000 P bbb0f321adbd492ab6df0b3d69b8bc9d [term 2 LEADER]: Becoming Leader. State: Replica: bbb0f321adbd492ab6df0b3d69b8bc9d, State: Running, Role: LEADER
I20260925 07:27:49.762061 18566 sys_catalog.cc:564] T 00000000000000000000000000000000 P bbb0f321adbd492ab6df0b3d69b8bc9d [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:27:49.762091 18570 consensus_queue.cc:237] T 00000000000000000000000000000000 P bbb0f321adbd492ab6df0b3d69b8bc9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbb0f321adbd492ab6df0b3d69b8bc9d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37921 } }
I20260925 07:27:49.762382 18570 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbb0f321adbd492ab6df0b3d69b8bc9d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "bbb0f321adbd492ab6df0b3d69b8bc9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbb0f321adbd492ab6df0b3d69b8bc9d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37921 } } }
I20260925 07:27:49.762522 18570 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbb0f321adbd492ab6df0b3d69b8bc9d [sys.catalog]: This master's current role is: LEADER
I20260925 07:27:49.762697 18570 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbb0f321adbd492ab6df0b3d69b8bc9d [sys.catalog]: SysCatalogTable state changed. Reason: New leader bbb0f321adbd492ab6df0b3d69b8bc9d. Latest consensus state: current_term: 2 leader_uuid: "bbb0f321adbd492ab6df0b3d69b8bc9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbb0f321adbd492ab6df0b3d69b8bc9d" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37921 } } }
I20260925 07:27:49.762806 18570 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbb0f321adbd492ab6df0b3d69b8bc9d [sys.catalog]: This master's current role is: LEADER
I20260925 07:27:49.762944 18578 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 07:27:49.763432 18578 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 07:27:49.763599 18578 catalog_manager.cc:1259] Loaded cluster ID: 333a3ff8d69247f3b5e0bc37a4997c55
I20260925 07:27:49.763676 18578 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 07:27:49.763720 18566 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260925 07:27:49.766499 18578 catalog_manager.cc:1504] Loading token signing keys...
I20260925 07:27:49.766810 18578 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bbb0f321adbd492ab6df0b3d69b8bc9d: Loaded TSK: 0
I20260925 07:27:49.767052 18578 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 07:27:49.873826 18568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:27:49.874195 18568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:27:49.874285 18568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:27:49.876044 18568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:27:49.876144 18568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:27:49.876312 18568 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.65
I20260925 07:27:49.881721 18568 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:45483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.158.84:52617/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab
--principal=kudu/127.5.158.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.65:44029
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.158.65
--webserver_port=37647
--tserver_master_addrs=127.5.158.126:37921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18568
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.65
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:27:49.882153 18568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:27:49.882447 18568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:27:49.888600 18603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:27:49.889230 18606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:27:49.889884 18604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:27:49.890137 18568 server_base.cc:1034] running on GCE node
I20260925 07:27:49.890439 18568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:27:49.890684 18568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:27:49.891872 18568 hybrid_clock.cc:648] HybridClock initialized: now 1790321269891850 us; error 35 us; skew 500 ppm
Sep 25 07:27:49 dist-test-slave-1drq krb5kdc[17534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321269, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:27:49.895987 18568 init.cc:377] Logged in from keytab as kudu/127.5.158.65@KRBTEST.COM (short username kudu)
I20260925 07:27:49.896445 18568 webserver.cc:460] Webserver started at http://127.5.158.65:37647/ using document root <none> and password file <none>
I20260925 07:27:49.896682 18568 fs_manager.cc:359] Metadata directory not provided
I20260925 07:27:49.896739 18568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 25 07:27:49 dist-test-slave-1drq krb5kdc[17534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321269, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.65@KRBTEST.COM for HTTP/127.5.158.84@KRBTEST.COM
I20260925 07:27:49.925464 18568 fs_manager.cc:709] Time spent opening directory manager: real 0.014s user 0.001s sys 0.000s
I20260925 07:27:49.926437 18614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:49.926661 18568 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260925 07:27:49.926744 18568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-0/wal
uuid: "d36767c07fe04ceab87767a829be1a28"
format_stamp: "Formatted at 2026-09-25 07:27:49 on dist-test-slave-1drq"
server_key: "a5e48257b6025b55ae778a8fc9e4cda9"
server_key_iv: "93eda88cfe4728d91acb419bcf146717"
server_key_version: "kuduclusterkey@0"
I20260925 07:27:49.927168 18568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:27:49.942983 18568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:27:49.945315 18568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:27:49.945520 18568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:27:49.945736 18568 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:27:49.946084 18568 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:27:49.946164 18568 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:49.946254 18568 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:27:49.946327 18568 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:49.952189 18568 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.65:44029
I20260925 07:27:49.952677 18568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
I20260925 07:27:49.955605 18727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.65:44029 every 8 connection(s)
I20260925 07:27:49.955893 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 18568
I20260925 07:27:49.956189 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.158.84:52617/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.66:38887
--local_ip_for_outbound_sockets=127.5.158.66
--tserver_master_addrs=127.5.158.126:37921
--webserver_port=41413
--webserver_interface=127.5.158.66
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab
--principal=kudu/127.5.158.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:45483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.158.84:52617/kms with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 07:27:49 dist-test-slave-1drq krb5kdc[17534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321269, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.65@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:27:49.972458 18730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:49.956230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37921 (local address 127.5.158.65:50751)
0925 07:27:49.956807 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:49.956848 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 07:27:49.957461 (+ 613us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:27:49.961680 (+ 4219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:27:49.961685 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:27:49.961908 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:27:49.962197 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:49.962203 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:49.963990 (+ 1787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:49.963991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:49.965351 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:49.965353 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:49.966216 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:49.966217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:49.966340 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:49.966903 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:27:49.966913 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:27:49.968557 (+ 1644us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:27:49.970777 (+ 2220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:27:49.970783 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:27:49.970785 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:27:49.971070 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:27:49.971337 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:27:49.971338 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:27:49.971338 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:27:49.971455 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:27:49.971897 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:27:49.971902 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:27:49.972038 (+ 136us) client_negotiation.cc:769] Sending connection context
0925 07:27:49.972117 (+ 79us) client_negotiation.cc:240] Negotiation successful
0925 07:27:49.972250 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":88,"threads_started":1}
I20260925 07:27:49.973438 18731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:49.956286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:50751 (local address 127.5.158.126:37921)
0925 07:27:49.961105 (+ 4819us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:49.961116 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:49.961157 (+ 41us) server_negotiation.cc:407] Connection header received
0925 07:27:49.961320 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:49.961328 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:49.961403 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:49.962335 (+ 932us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:27:49.962424 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:49.963859 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:49.965465 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:49.966084 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:49.966594 (+ 510us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:49.968668 (+ 2074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:27:49.968672 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:27:49.968673 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:27:49.968683 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:27:49.970677 (+ 1994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:49.971162 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:49.971165 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:49.971165 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:49.971264 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:49.971523 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:49.971524 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:49.971524 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:49.971779 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:27:49.973017 (+ 1238us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:49.973130 (+ 113us) server_negotiation.cc:299] Negotiation successful
0925 07:27:49.973246 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4748,"thread_start_us":71,"threads_started":1}
I20260925 07:27:49.974512 18728 heartbeater.cc:347] Connected to a master server at 127.5.158.126:37921
I20260925 07:27:49.974602 18728 heartbeater.cc:464] Registering TS with master...
I20260925 07:27:49.974776 18728 heartbeater.cc:511] Master 127.5.158.126:37921 requested a full tablet report, sending...
W20260925 07:27:49.975625 18728 heartbeater.cc:644] Failed to heartbeat to 127.5.158.126:37921 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260925 07:27:49.975684 18728 heartbeater.cc:464] Registering TS with master...
I20260925 07:27:49.975736 18728 heartbeater.cc:511] Master 127.5.158.126:37921 requested a full tablet report, sending...
I20260925 07:27:49.976123 18728 heartbeater.cc:464] Registering TS with master...
I20260925 07:27:49.976195 18728 heartbeater.cc:511] Master 127.5.158.126:37921 requested a full tablet report, sending...
W20260925 07:27:49.976500 18728 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 07:27:50.066447 18732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:27:50.066772 18732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:27:50.066850 18732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:27:50.068444 18732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:27:50.068555 18732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:27:50.068635 18732 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.66
I20260925 07:27:50.070864 18732 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:45483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.158.84:52617/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab
--principal=kudu/127.5.158.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.66:38887
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.158.66
--webserver_port=41413
--tserver_master_addrs=127.5.158.126:37921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18732
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.66
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:27:50.071297 18732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:27:50.071625 18732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:27:50.077910 18738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:27:50.078217 18740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:27:50.078819 18737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:27:50.079597 18732 server_base.cc:1034] running on GCE node
I20260925 07:27:50.079890 18732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:27:50.080137 18732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:27:50.083410 18732 hybrid_clock.cc:648] HybridClock initialized: now 1790321270082741 us; error 677 us; skew 500 ppm
Sep 25 07:27:50 dist-test-slave-1drq krb5kdc[17534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321270, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:27:50.086474 18732 init.cc:377] Logged in from keytab as kudu/127.5.158.66@KRBTEST.COM (short username kudu)
I20260925 07:27:50.086935 18732 webserver.cc:460] Webserver started at http://127.5.158.66:41413/ using document root <none> and password file <none>
I20260925 07:27:50.087198 18732 fs_manager.cc:359] Metadata directory not provided
I20260925 07:27:50.087262 18732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 25 07:27:50 dist-test-slave-1drq krb5kdc[17534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321270, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.66@KRBTEST.COM for HTTP/127.5.158.84@KRBTEST.COM
I20260925 07:27:50.117022 18732 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.001s sys 0.000s
I20260925 07:27:50.118036 18748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:50.118220 18732 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260925 07:27:50.118294 18732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-1/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-1/wal
uuid: "2f41581887534ad29bb86fbeee06df42"
format_stamp: "Formatted at 2026-09-25 07:27:49 on dist-test-slave-1drq"
server_key: "cb47d2d927588e0062a72f3c1a7f5427"
server_key_iv: "66381b457f976276c4ad717ffe7739a6"
server_key_version: "kuduclusterkey@0"
I20260925 07:27:50.118732 18732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:27:50.130952 18732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:27:50.133244 18732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:27:50.133424 18732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:27:50.133672 18732 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:27:50.134073 18732 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:27:50.134150 18732 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:50.134264 18732 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:27:50.134315 18732 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:50.141819 18861 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.66:38887 every 8 connection(s)
I20260925 07:27:50.141937 18732 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.66:38887
I20260925 07:27:50.142500 18732 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
I20260925 07:27:50.144450 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 18732
I20260925 07:27:50.144761 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.158.84:52617/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.67:43091
--local_ip_for_outbound_sockets=127.5.158.67
--tserver_master_addrs=127.5.158.126:37921
--webserver_port=45065
--webserver_interface=127.5.158.67
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab
--principal=kudu/127.5.158.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:45483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.158.84:52617/kms with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 07:27:50 dist-test-slave-1drq krb5kdc[17534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321270, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.66@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:27:50.159068 18731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:50.144122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:47329 (local address 127.5.158.126:37921)
0925 07:27:50.144166 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:50.144169 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:50.146647 (+ 2478us) server_negotiation.cc:407] Connection header received
0925 07:27:50.147344 (+ 697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:50.147346 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:50.147358 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:50.147391 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:27:50.148181 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:50.149056 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:50.150865 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:50.151242 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:50.151288 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:50.155741 (+ 4453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:27:50.155743 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:27:50.155743 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:27:50.155750 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:27:50.157484 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:50.157988 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:50.157989 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:50.157990 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:50.158047 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:50.158335 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:50.158336 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:50.158337 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:50.158468 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:27:50.158511 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:50.158884 (+ 373us) server_negotiation.cc:299] Negotiation successful
0925 07:27:50.158970 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 07:27:50.159811 18864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:50.143852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37921 (local address 127.5.158.66:47329)
0925 07:27:50.144253 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:50.146557 (+ 2304us) client_negotiation.cc:174] Beginning negotiation
0925 07:27:50.147213 (+ 656us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:27:50.147569 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:27:50.147573 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:27:50.147799 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:27:50.148066 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:50.148073 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:50.149224 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:50.149225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:50.150740 (+ 1515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:50.150743 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:50.151402 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:50.151403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:50.151543 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:50.153234 (+ 1691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:27:50.153244 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:27:50.155627 (+ 2383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:27:50.157595 (+ 1968us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:27:50.157600 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:27:50.157602 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:27:50.157897 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:27:50.158135 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:27:50.158135 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:27:50.158136 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:27:50.158258 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:27:50.158558 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:27:50.158563 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:27:50.158715 (+ 152us) client_negotiation.cc:769] Sending connection context
0925 07:27:50.159415 (+ 700us) client_negotiation.cc:240] Negotiation successful
0925 07:27:50.159573 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":79,"threads_started":1}
I20260925 07:27:50.160449 18862 heartbeater.cc:347] Connected to a master server at 127.5.158.126:37921
I20260925 07:27:50.160530 18862 heartbeater.cc:464] Registering TS with master...
I20260925 07:27:50.160697 18862 heartbeater.cc:511] Master 127.5.158.126:37921 requested a full tablet report, sending...
W20260925 07:27:50.161396 18862 heartbeater.cc:644] Failed to heartbeat to 127.5.158.126:37921 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260925 07:27:50.161458 18862 heartbeater.cc:464] Registering TS with master...
I20260925 07:27:50.161535 18862 heartbeater.cc:511] Master 127.5.158.126:37921 requested a full tablet report, sending...
I20260925 07:27:50.161892 18862 heartbeater.cc:464] Registering TS with master...
I20260925 07:27:50.161962 18862 heartbeater.cc:511] Master 127.5.158.126:37921 requested a full tablet report, sending...
W20260925 07:27:50.162266 18862 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 07:27:50.268090 18865 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:27:50.268332 18865 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:27:50.268364 18865 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:27:50.269449 18865 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:27:50.269488 18865 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:27:50.269513 18865 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.67
I20260925 07:27:50.270860 18865 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:45483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.158.84:52617/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab
--principal=kudu/127.5.158.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.67:43091
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.158.67
--webserver_port=45065
--tserver_master_addrs=127.5.158.126:37921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18865
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.67
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:27:50.271066 18865 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:27:50.271279 18865 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:27:50.277619 18873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:27:50.277969 18871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:27:50.278103 18865 server_base.cc:1034] running on GCE node
W20260925 07:27:50.278513 18870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:27:50.278805 18865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:27:50.279047 18865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:27:50.280207 18865 hybrid_clock.cc:648] HybridClock initialized: now 1790321270280171 us; error 50 us; skew 500 ppm
Sep 25 07:27:50 dist-test-slave-1drq krb5kdc[17534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321270, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:27:50.283373 18865 init.cc:377] Logged in from keytab as kudu/127.5.158.67@KRBTEST.COM (short username kudu)
I20260925 07:27:50.283854 18865 webserver.cc:460] Webserver started at http://127.5.158.67:45065/ using document root <none> and password file <none>
I20260925 07:27:50.284101 18865 fs_manager.cc:359] Metadata directory not provided
I20260925 07:27:50.284163 18865 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 25 07:27:50 dist-test-slave-1drq krb5kdc[17534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321270, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.67@KRBTEST.COM for HTTP/127.5.158.84@KRBTEST.COM
I20260925 07:27:50.298825 18865 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 07:27:50.299733 18881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:50.299923 18865 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260925 07:27:50.299994 18865 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-2/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-2/wal
uuid: "8c9a642ca63546dda148f11f2974bf01"
format_stamp: "Formatted at 2026-09-25 07:27:49 on dist-test-slave-1drq"
server_key: "2389e88be6adcb082b199a62c31ea4d1"
server_key_iv: "bfe2856ea2e318a39719ebb95e02429e"
server_key_version: "kuduclusterkey@0"
I20260925 07:27:50.300386 18865 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:27:50.315665 18865 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:27:50.317833 18865 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:27:50.317988 18865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:27:50.318203 18865 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:27:50.318548 18865 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:27:50.318614 18865 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:50.318655 18865 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:27:50.318683 18865 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:50.326620 18865 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.67:43091
I20260925 07:27:50.326728 18995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.67:43091 every 8 connection(s)
I20260925 07:27:50.327688 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 18865
I20260925 07:27:50.327070 18865 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
I20260925 07:27:50.335841 18731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:50.328332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47664 (local address 127.5.158.126:37921)
0925 07:27:50.328384 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:50.328386 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:50.328396 (+ 10us) server_negotiation.cc:407] Connection header received
0925 07:27:50.328424 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:50.328425 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:50.328437 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:50.328484 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:27:50.329266 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:50.329850 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:50.330720 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:50.330973 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:50.331062 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:50.332594 (+ 1532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:27:50.332596 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:27:50.332596 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:27:50.332604 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:27:50.334432 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:50.334881 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:50.334882 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:50.334883 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:50.334942 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:50.335158 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:50.335158 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:50.335159 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:50.335275 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:27:50.335409 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:50.335643 (+ 234us) server_negotiation.cc:299] Negotiation successful
0925 07:27:50.335728 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
Sep 25 07:27:50 dist-test-slave-1drq krb5kdc[17534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321270, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.67@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:27:50.343727 18999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:50.328624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37921 (local address 127.5.158.67:44499)
0925 07:27:50.330364 (+ 1740us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:50.330410 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0925 07:27:50.331505 (+ 1095us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:27:50.331941 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:27:50.331945 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:27:50.332150 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:27:50.332377 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:50.332383 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:50.333342 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:50.333343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:50.334586 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:50.334589 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:50.337703 (+ 3114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:50.337704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:50.337839 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:50.338354 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:27:50.338363 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:27:50.339874 (+ 1511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:27:50.341882 (+ 2008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:27:50.341887 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:27:50.341889 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:27:50.342088 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:27:50.342804 (+ 716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:27:50.342805 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:27:50.342805 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:27:50.342916 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:27:50.343196 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:27:50.343201 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:27:50.343326 (+ 125us) client_negotiation.cc:769] Sending connection context
0925 07:27:50.343448 (+ 122us) client_negotiation.cc:240] Negotiation successful
0925 07:27:50.343579 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1668,"thread_start_us":83,"threads_started":1}
I20260925 07:27:50.344059 19000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:50.328926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:44499 (local address 127.5.158.126:37921)
0925 07:27:50.331714 (+ 2788us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:50.331716 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:50.331723 (+ 7us) server_negotiation.cc:407] Connection header received
0925 07:27:50.331757 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:50.331757 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:50.331773 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:50.332476 (+ 703us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:27:50.332513 (+ 37us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:50.333216 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:50.337304 (+ 4088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:50.337583 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:50.337665 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:50.339980 (+ 2315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:27:50.339981 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:27:50.339981 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:27:50.339988 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:27:50.341728 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:50.342649 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:50.342650 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:50.342650 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:50.342704 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:50.342986 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:50.342987 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:50.342987 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:50.343108 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:27:50.343192 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:50.343900 (+ 708us) server_negotiation.cc:299] Negotiation successful
0925 07:27:50.343983 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2761,"thread_start_us":83,"threads_started":1}
I20260925 07:27:50.344373 18996 heartbeater.cc:347] Connected to a master server at 127.5.158.126:37921
I20260925 07:27:50.344444 18996 heartbeater.cc:464] Registering TS with master...
I20260925 07:27:50.344563 18996 heartbeater.cc:511] Master 127.5.158.126:37921 requested a full tablet report, sending...
W20260925 07:27:50.345217 18996 heartbeater.cc:644] Failed to heartbeat to 127.5.158.126:37921 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260925 07:27:50.345269 18996 heartbeater.cc:464] Registering TS with master...
I20260925 07:27:50.345309 18996 heartbeater.cc:511] Master 127.5.158.126:37921 requested a full tablet report, sending...
I20260925 07:27:50.345727 18996 heartbeater.cc:464] Registering TS with master...
I20260925 07:27:50.345808 18996 heartbeater.cc:511] Master 127.5.158.126:37921 requested a full tablet report, sending...
W20260925 07:27:50.346123 18996 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 25 07:27:50 dist-test-slave-1drq krb5kdc[17534](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790321270, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:27:50.976681 18728 heartbeater.cc:464] Registering TS with master...
I20260925 07:27:50.976851 18728 heartbeater.cc:511] Master 127.5.158.126:37921 requested a full tablet report, sending...
I20260925 07:27:50.977257 18530 ts_manager.cc:194] Registered new tserver with Master: d36767c07fe04ceab87767a829be1a28 (127.5.158.65:44029)
I20260925 07:27:50.978235 18530 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.65@KRBTEST.COM'} at 127.5.158.65:50751
I20260925 07:27:51.162412 18862 heartbeater.cc:464] Registering TS with master...
I20260925 07:27:51.162547 18862 heartbeater.cc:511] Master 127.5.158.126:37921 requested a full tablet report, sending...
I20260925 07:27:51.162772 18530 ts_manager.cc:194] Registered new tserver with Master: 2f41581887534ad29bb86fbeee06df42 (127.5.158.66:38887)
I20260925 07:27:51.163254 18530 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.66@KRBTEST.COM'} at 127.5.158.66:47329
Sep 25 07:27:51 dist-test-slave-1drq krb5kdc[17534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321270, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.126@KRBTEST.COM for HTTP/127.5.158.84@KRBTEST.COM
I20260925 07:27:51.346319 18996 heartbeater.cc:464] Registering TS with master...
I20260925 07:27:51.346472 18996 heartbeater.cc:511] Master 127.5.158.126:37921 requested a full tablet report, sending...
I20260925 07:27:51.347164 18530 ts_manager.cc:194] Registered new tserver with Master: 8c9a642ca63546dda148f11f2974bf01 (127.5.158.67:43091)
I20260925 07:27:51.347659 18530 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.67@KRBTEST.COM'} at 127.5.158.67:44499
I20260925 07:27:51.355631 5753 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 07:27:51.364260 19018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:51.356552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47668 (local address 127.5.158.126:37921)
0925 07:27:51.356864 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:51.356866 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:51.356873 (+ 7us) server_negotiation.cc:407] Connection header received
0925 07:27:51.356895 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:51.356896 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:51.356908 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:51.356982 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:27:51.357593 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:51.358372 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:51.359339 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:51.359674 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:51.359749 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:51.361059 (+ 1310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:27:51.361061 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:27:51.361061 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:27:51.361068 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:27:51.362932 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:51.363350 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:51.363351 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:51.363352 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:51.363418 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:51.363656 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:51.363656 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:51.363657 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:51.363795 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:27:51.363902 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:51.364065 (+ 163us) server_negotiation.cc:299] Negotiation successful
0925 07:27:51.364155 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":62,"threads_started":1}
I20260925 07:27:51.367008 18530 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47668:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 07:27:51.369256 18530 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 07:27:51.380136 19024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:51.375178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:44029 (local address 127.0.0.1:33414)
0925 07:27:51.375548 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:51.375583 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 07:27:51.375635 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:27:51.376122 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:27:51.376125 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:27:51.376127 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:27:51.376177 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:51.376178 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:51.377324 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:51.377325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:51.378734 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:51.378737 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:51.379671 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:51.379672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:51.379737 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:51.379748 (+ 11us) client_negotiation.cc:769] Sending connection context
0925 07:27:51.379804 (+ 56us) client_negotiation.cc:240] Negotiation successful
0925 07:27:51.379847 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":102,"threads_started":1}
I20260925 07:27:51.381155 19027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:51.375442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47052 (local address 127.5.158.67:43091)
0925 07:27:51.375779 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:51.375783 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:51.377254 (+ 1471us) server_negotiation.cc:407] Connection header received
0925 07:27:51.377391 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:51.377394 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:51.377408 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:51.377460 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:27:51.377716 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:51.378714 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:51.379948 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:51.380692 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:51.380743 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:51.380793 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:51.381010 (+ 217us) server_negotiation.cc:299] Negotiation successful
0925 07:27:51.381051 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":57,"threads_started":1}
I20260925 07:27:51.381381 19023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:51.375182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.67:43091 (local address 127.0.0.1:47052)
0925 07:27:51.377181 (+ 1999us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:51.377191 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 07:27:51.377305 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:27:51.377571 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:27:51.377572 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:27:51.377574 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:27:51.377616 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:51.377617 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:51.378827 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:51.378828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:51.379839 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:51.379842 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:51.380856 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:51.380857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:51.380916 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:51.380920 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 07:27:51.381274 (+ 354us) client_negotiation.cc:240] Negotiation successful
0925 07:27:51.381314 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1982,"thread_start_us":99,"threads_started":1}
I20260925 07:27:51.382037 18930 tablet_service.cc:1467] Processing CreateTablet for tablet b92c4fa2a41a4bebaab5fd9ca4556973 (DEFAULT_TABLE table=test-table [id=8fa09b3b525748a2a2bc7fceedb7f5d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:27:51.382402 18930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b92c4fa2a41a4bebaab5fd9ca4556973. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:27:51.384384 19028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:51.375569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50502 (local address 127.5.158.66:38887)
0925 07:27:51.375719 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:51.375723 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:51.380462 (+ 4739us) server_negotiation.cc:407] Connection header received
0925 07:27:51.380577 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:51.380579 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:51.380593 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:51.380637 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:27:51.380815 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:51.381777 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:51.383182 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:51.383952 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:51.384001 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:51.384047 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:51.384269 (+ 222us) server_negotiation.cc:299] Negotiation successful
0925 07:27:51.384307 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":49,"threads_started":1}
I20260925 07:27:51.384596 19024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:51.375421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.66:38887 (local address 127.0.0.1:50502)
0925 07:27:51.380408 (+ 4987us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:51.380416 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 07:27:51.380490 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:27:51.380728 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:27:51.380729 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:27:51.380730 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:27:51.380755 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:51.380756 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:51.381866 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:51.381866 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:51.383076 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:51.383079 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:51.384105 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:51.384105 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:51.384179 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:51.384181 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 07:27:51.384497 (+ 316us) client_negotiation.cc:240] Negotiation successful
0925 07:27:51.384536 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4977,"thread_start_us":53,"threads_started":1}
I20260925 07:27:51.385064 19025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:51.375277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33414 (local address 127.5.158.65:44029)
0925 07:27:51.375878 (+ 601us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:51.375883 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:51.375891 (+ 8us) server_negotiation.cc:407] Connection header received
0925 07:27:51.375923 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:51.375925 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:51.375940 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:51.376046 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:27:51.376257 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:51.377220 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:51.378828 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:51.379533 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:51.379593 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:51.379641 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:51.384917 (+ 5276us) server_negotiation.cc:299] Negotiation successful
0925 07:27:51.384972 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":45,"threads_started":1}
I20260925 07:27:51.385418 18782 tablet_service.cc:1467] Processing CreateTablet for tablet b92c4fa2a41a4bebaab5fd9ca4556973 (DEFAULT_TABLE table=test-table [id=8fa09b3b525748a2a2bc7fceedb7f5d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:27:51.385632 18662 tablet_service.cc:1467] Processing CreateTablet for tablet b92c4fa2a41a4bebaab5fd9ca4556973 (DEFAULT_TABLE table=test-table [id=8fa09b3b525748a2a2bc7fceedb7f5d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:27:51.385761 18782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b92c4fa2a41a4bebaab5fd9ca4556973. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:27:51.385931 18662 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b92c4fa2a41a4bebaab5fd9ca4556973. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:27:51.388075 19030 tablet_bootstrap.cc:492] T b92c4fa2a41a4bebaab5fd9ca4556973 P 2f41581887534ad29bb86fbeee06df42: Bootstrap starting.
I20260925 07:27:51.388640 19030 tablet_bootstrap.cc:654] T b92c4fa2a41a4bebaab5fd9ca4556973 P 2f41581887534ad29bb86fbeee06df42: Neither blocks nor log segments found. Creating new log.
I20260925 07:27:51.388952 19030 log.cc:824] T b92c4fa2a41a4bebaab5fd9ca4556973 P 2f41581887534ad29bb86fbeee06df42: Log is configured to *not* fsync() on all Append() calls
I20260925 07:27:51.393210 19029 tablet_bootstrap.cc:492] T b92c4fa2a41a4bebaab5fd9ca4556973 P 8c9a642ca63546dda148f11f2974bf01: Bootstrap starting.
I20260925 07:27:51.393473 19031 tablet_bootstrap.cc:492] T b92c4fa2a41a4bebaab5fd9ca4556973 P d36767c07fe04ceab87767a829be1a28: Bootstrap starting.
I20260925 07:27:51.393710 19029 tablet_bootstrap.cc:654] T b92c4fa2a41a4bebaab5fd9ca4556973 P 8c9a642ca63546dda148f11f2974bf01: Neither blocks nor log segments found. Creating new log.
I20260925 07:27:51.393993 19029 log.cc:824] T b92c4fa2a41a4bebaab5fd9ca4556973 P 8c9a642ca63546dda148f11f2974bf01: Log is configured to *not* fsync() on all Append() calls
I20260925 07:27:51.394193 19031 tablet_bootstrap.cc:654] T b92c4fa2a41a4bebaab5fd9ca4556973 P d36767c07fe04ceab87767a829be1a28: Neither blocks nor log segments found. Creating new log.
I20260925 07:27:51.394500 19031 log.cc:824] T b92c4fa2a41a4bebaab5fd9ca4556973 P d36767c07fe04ceab87767a829be1a28: Log is configured to *not* fsync() on all Append() calls
I20260925 07:27:51.394692 19030 tablet_bootstrap.cc:492] T b92c4fa2a41a4bebaab5fd9ca4556973 P 2f41581887534ad29bb86fbeee06df42: No bootstrap required, opened a new log
I20260925 07:27:51.394759 19030 ts_tablet_manager.cc:1397] T b92c4fa2a41a4bebaab5fd9ca4556973 P 2f41581887534ad29bb86fbeee06df42: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.001s
I20260925 07:27:51.395203 19031 tablet_bootstrap.cc:492] T b92c4fa2a41a4bebaab5fd9ca4556973 P d36767c07fe04ceab87767a829be1a28: No bootstrap required, opened a new log
I20260925 07:27:51.395270 19031 ts_tablet_manager.cc:1397] T b92c4fa2a41a4bebaab5fd9ca4556973 P d36767c07fe04ceab87767a829be1a28: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 07:27:51.396929 19030 raft_consensus.cc:348] T b92c4fa2a41a4bebaab5fd9ca4556973 P 2f41581887534ad29bb86fbeee06df42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c9a642ca63546dda148f11f2974bf01" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 43091 } } peers { permanent_uuid: "d36767c07fe04ceab87767a829be1a28" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 44029 } } peers { permanent_uuid: "2f41581887534ad29bb86fbeee06df42" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 38887 } }
I20260925 07:27:51.397132 19030 raft_consensus.cc:374] T b92c4fa2a41a4bebaab5fd9ca4556973 P 2f41581887534ad29bb86fbeee06df42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:27:51.397179 19030 raft_consensus.cc:729] T b92c4fa2a41a4bebaab5fd9ca4556973 P 2f41581887534ad29bb86fbeee06df42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f41581887534ad29bb86fbeee06df42, State: Initialized, Role: FOLLOWER
I20260925 07:27:51.397300 19030 consensus_queue.cc:260] T b92c4fa2a41a4bebaab5fd9ca4556973 P 2f41581887534ad29bb86fbeee06df42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c9a642ca63546dda148f11f2974bf01" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 43091 } } peers { permanent_uuid: "d36767c07fe04ceab87767a829be1a28" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 44029 } } peers { permanent_uuid: "2f41581887534ad29bb86fbeee06df42" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 38887 } }
I20260925 07:27:51.397535 19030 ts_tablet_manager.cc:1428] T b92c4fa2a41a4bebaab5fd9ca4556973 P 2f41581887534ad29bb86fbeee06df42: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260925 07:27:51.397632 18862 heartbeater.cc:503] Master 127.5.158.126:37921 was elected leader, sending a full tablet report...
I20260925 07:27:51.397840 19031 raft_consensus.cc:348] T b92c4fa2a41a4bebaab5fd9ca4556973 P d36767c07fe04ceab87767a829be1a28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c9a642ca63546dda148f11f2974bf01" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 43091 } } peers { permanent_uuid: "d36767c07fe04ceab87767a829be1a28" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 44029 } } peers { permanent_uuid: "2f41581887534ad29bb86fbeee06df42" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 38887 } }
I20260925 07:27:51.397995 19031 raft_consensus.cc:374] T b92c4fa2a41a4bebaab5fd9ca4556973 P d36767c07fe04ceab87767a829be1a28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:27:51.398032 19031 raft_consensus.cc:729] T b92c4fa2a41a4bebaab5fd9ca4556973 P d36767c07fe04ceab87767a829be1a28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d36767c07fe04ceab87767a829be1a28, State: Initialized, Role: FOLLOWER
I20260925 07:27:51.398140 19031 consensus_queue.cc:260] T b92c4fa2a41a4bebaab5fd9ca4556973 P d36767c07fe04ceab87767a829be1a28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c9a642ca63546dda148f11f2974bf01" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 43091 } } peers { permanent_uuid: "d36767c07fe04ceab87767a829be1a28" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 44029 } } peers { permanent_uuid: "2f41581887534ad29bb86fbeee06df42" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 38887 } }
I20260925 07:27:51.398636 19031 ts_tablet_manager.cc:1428] T b92c4fa2a41a4bebaab5fd9ca4556973 P d36767c07fe04ceab87767a829be1a28: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 07:27:51.398757 18728 heartbeater.cc:503] Master 127.5.158.126:37921 was elected leader, sending a full tablet report...
I20260925 07:27:51.399071 19029 tablet_bootstrap.cc:492] T b92c4fa2a41a4bebaab5fd9ca4556973 P 8c9a642ca63546dda148f11f2974bf01: No bootstrap required, opened a new log
I20260925 07:27:51.399133 19029 ts_tablet_manager.cc:1397] T b92c4fa2a41a4bebaab5fd9ca4556973 P 8c9a642ca63546dda148f11f2974bf01: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20260925 07:27:51.401228 19029 raft_consensus.cc:348] T b92c4fa2a41a4bebaab5fd9ca4556973 P 8c9a642ca63546dda148f11f2974bf01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c9a642ca63546dda148f11f2974bf01" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 43091 } } peers { permanent_uuid: "d36767c07fe04ceab87767a829be1a28" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 44029 } } peers { permanent_uuid: "2f41581887534ad29bb86fbeee06df42" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 38887 } }
I20260925 07:27:51.401360 19029 raft_consensus.cc:374] T b92c4fa2a41a4bebaab5fd9ca4556973 P 8c9a642ca63546dda148f11f2974bf01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:27:51.401384 19029 raft_consensus.cc:729] T b92c4fa2a41a4bebaab5fd9ca4556973 P 8c9a642ca63546dda148f11f2974bf01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c9a642ca63546dda148f11f2974bf01, State: Initialized, Role: FOLLOWER
I20260925 07:27:51.401500 19029 consensus_queue.cc:260] T b92c4fa2a41a4bebaab5fd9ca4556973 P 8c9a642ca63546dda148f11f2974bf01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c9a642ca63546dda148f11f2974bf01" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 43091 } } peers { permanent_uuid: "d36767c07fe04ceab87767a829be1a28" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 44029 } } peers { permanent_uuid: "2f41581887534ad29bb86fbeee06df42" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 38887 } }
I20260925 07:27:51.401707 19029 ts_tablet_manager.cc:1428] T b92c4fa2a41a4bebaab5fd9ca4556973 P 8c9a642ca63546dda148f11f2974bf01: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260925 07:27:51.401835 18996 heartbeater.cc:503] Master 127.5.158.126:37921 was elected leader, sending a full tablet report...
Sep 25 07:27:51 dist-test-slave-1drq krb5kdc[17534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321270, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.126@KRBTEST.COM for HTTP/127.5.158.84@KRBTEST.COM
W20260925 07:27:51.457593 18729 tablet.cc:2369] T b92c4fa2a41a4bebaab5fd9ca4556973 P d36767c07fe04ceab87767a829be1a28: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 07:27:51.465240 19037 raft_consensus.cc:482] T b92c4fa2a41a4bebaab5fd9ca4556973 P 8c9a642ca63546dda148f11f2974bf01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 07:27:51.465369 19037 raft_consensus.cc:504] T b92c4fa2a41a4bebaab5fd9ca4556973 P 8c9a642ca63546dda148f11f2974bf01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c9a642ca63546dda148f11f2974bf01" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 43091 } } peers { permanent_uuid: "d36767c07fe04ceab87767a829be1a28" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 44029 } } peers { permanent_uuid: "2f41581887534ad29bb86fbeee06df42" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 38887 } }
I20260925 07:27:51.465775 19037 leader_election.cc:288] T b92c4fa2a41a4bebaab5fd9ca4556973 P 8c9a642ca63546dda148f11f2974bf01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d36767c07fe04ceab87767a829be1a28 (127.5.158.65:44029), 2f41581887534ad29bb86fbeee06df42 (127.5.158.66:38887)
I20260925 07:27:51.469940 19028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:51.466173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:36217 (local address 127.5.158.66:38887)
0925 07:27:51.466218 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:51.466221 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:51.467038 (+ 817us) server_negotiation.cc:407] Connection header received
0925 07:27:51.467128 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:51.467129 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:51.467138 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:51.467162 (+ 24us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:27:51.467321 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:51.467900 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:51.469068 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:51.469572 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:51.469611 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:51.469619 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:51.469834 (+ 215us) server_negotiation.cc:299] Negotiation successful
0925 07:27:51.469881 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 07:27:51.469982 19025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:51.466152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:56203 (local address 127.5.158.65:44029)
0925 07:27:51.466199 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:51.466201 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:51.466749 (+ 548us) server_negotiation.cc:407] Connection header received
0925 07:27:51.466822 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:51.466823 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:51.466833 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:51.466881 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:27:51.467076 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:51.467735 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:51.469212 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:51.469678 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:51.469714 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:51.469721 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:51.469889 (+ 168us) server_negotiation.cc:299] Negotiation successful
0925 07:27:51.469924 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 07:27:51.470180 19039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:51.465945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.66:38887 (local address 127.5.158.67:36217)
0925 07:27:51.466846 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:51.466853 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0925 07:27:51.467061 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:27:51.467213 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:27:51.467214 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:27:51.467215 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:27:51.467261 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:51.467263 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:51.467971 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:51.467972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:51.468982 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:51.468985 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:51.469695 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:51.469696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:51.469762 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:51.469765 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 07:27:51.470051 (+ 286us) client_negotiation.cc:240] Negotiation successful
0925 07:27:51.470102 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":886,"mutex_wait_us":11,"thread_start_us":77,"threads_started":1}
I20260925 07:27:51.470180 19040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:51.465903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:44029 (local address 127.5.158.67:56203)
0925 07:27:51.466679 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:51.466688 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 07:27:51.466758 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:27:51.466926 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:27:51.466927 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:27:51.466929 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:27:51.467000 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:51.467002 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:51.467834 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:51.467835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:51.469127 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:51.469129 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:51.469764 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:51.469765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:51.469825 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:51.469828 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 07:27:51.470061 (+ 233us) client_negotiation.cc:240] Negotiation successful
0925 07:27:51.470092 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":760,"thread_start_us":88,"threads_started":1}
I20260925 07:27:51.470463 18816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b92c4fa2a41a4bebaab5fd9ca4556973" candidate_uuid: "8c9a642ca63546dda148f11f2974bf01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f41581887534ad29bb86fbeee06df42" is_pre_election: true
I20260925 07:27:51.470485 18682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b92c4fa2a41a4bebaab5fd9ca4556973" candidate_uuid: "8c9a642ca63546dda148f11f2974bf01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d36767c07fe04ceab87767a829be1a28" is_pre_election: true
I20260925 07:27:51.470607 18816 raft_consensus.cc:2454] T b92c4fa2a41a4bebaab5fd9ca4556973 P 2f41581887534ad29bb86fbeee06df42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c9a642ca63546dda148f11f2974bf01 in term 0.
I20260925 07:27:51.470649 18682 raft_consensus.cc:2454] T b92c4fa2a41a4bebaab5fd9ca4556973 P d36767c07fe04ceab87767a829be1a28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c9a642ca63546dda148f11f2974bf01 in term 0.
I20260925 07:27:51.470824 18884 leader_election.cc:302] T b92c4fa2a41a4bebaab5fd9ca4556973 P 8c9a642ca63546dda148f11f2974bf01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f41581887534ad29bb86fbeee06df42, 8c9a642ca63546dda148f11f2974bf01; no voters:
I20260925 07:27:51.470952 19037 raft_consensus.cc:2781] T b92c4fa2a41a4bebaab5fd9ca4556973 P 8c9a642ca63546dda148f11f2974bf01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 07:27:51.470992 19037 raft_consensus.cc:482] T b92c4fa2a41a4bebaab5fd9ca4556973 P 8c9a642ca63546dda148f11f2974bf01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 07:27:51.471016 19037 raft_consensus.cc:3037] T b92c4fa2a41a4bebaab5fd9ca4556973 P 8c9a642ca63546dda148f11f2974bf01 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:27:51.471618 19037 raft_consensus.cc:504] T b92c4fa2a41a4bebaab5fd9ca4556973 P 8c9a642ca63546dda148f11f2974bf01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c9a642ca63546dda148f11f2974bf01" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 43091 } } peers { permanent_uuid: "d36767c07fe04ceab87767a829be1a28" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 44029 } } peers { permanent_uuid: "2f41581887534ad29bb86fbeee06df42" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 38887 } }
I20260925 07:27:51.471743 19037 leader_election.cc:288] T b92c4fa2a41a4bebaab5fd9ca4556973 P 8c9a642ca63546dda148f11f2974bf01 [CANDIDATE]: Term 1 election: Requested vote from peers d36767c07fe04ceab87767a829be1a28 (127.5.158.65:44029), 2f41581887534ad29bb86fbeee06df42 (127.5.158.66:38887)
I20260925 07:27:51.471885 18816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b92c4fa2a41a4bebaab5fd9ca4556973" candidate_uuid: "8c9a642ca63546dda148f11f2974bf01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f41581887534ad29bb86fbeee06df42"
I20260925 07:27:51.471920 18682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b92c4fa2a41a4bebaab5fd9ca4556973" candidate_uuid: "8c9a642ca63546dda148f11f2974bf01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d36767c07fe04ceab87767a829be1a28"
I20260925 07:27:51.471949 18816 raft_consensus.cc:3037] T b92c4fa2a41a4bebaab5fd9ca4556973 P 2f41581887534ad29bb86fbeee06df42 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:27:51.471973 18682 raft_consensus.cc:3037] T b92c4fa2a41a4bebaab5fd9ca4556973 P d36767c07fe04ceab87767a829be1a28 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:27:51.472707 18682 raft_consensus.cc:2454] T b92c4fa2a41a4bebaab5fd9ca4556973 P d36767c07fe04ceab87767a829be1a28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c9a642ca63546dda148f11f2974bf01 in term 1.
I20260925 07:27:51.472703 18816 raft_consensus.cc:2454] T b92c4fa2a41a4bebaab5fd9ca4556973 P 2f41581887534ad29bb86fbeee06df42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c9a642ca63546dda148f11f2974bf01 in term 1.
I20260925 07:27:51.472846 18884 leader_election.cc:302] T b92c4fa2a41a4bebaab5fd9ca4556973 P 8c9a642ca63546dda148f11f2974bf01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f41581887534ad29bb86fbeee06df42, 8c9a642ca63546dda148f11f2974bf01; no voters:
I20260925 07:27:51.472952 19037 raft_consensus.cc:2781] T b92c4fa2a41a4bebaab5fd9ca4556973 P 8c9a642ca63546dda148f11f2974bf01 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:27:51.473110 19037 raft_consensus.cc:686] T b92c4fa2a41a4bebaab5fd9ca4556973 P 8c9a642ca63546dda148f11f2974bf01 [term 1 LEADER]: Becoming Leader. State: Replica: 8c9a642ca63546dda148f11f2974bf01, State: Running, Role: LEADER
I20260925 07:27:51.473203 19037 consensus_queue.cc:237] T b92c4fa2a41a4bebaab5fd9ca4556973 P 8c9a642ca63546dda148f11f2974bf01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c9a642ca63546dda148f11f2974bf01" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 43091 } } peers { permanent_uuid: "d36767c07fe04ceab87767a829be1a28" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 44029 } } peers { permanent_uuid: "2f41581887534ad29bb86fbeee06df42" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 38887 } }
I20260925 07:27:51.473970 18530 catalog_manager.cc:5495] T b92c4fa2a41a4bebaab5fd9ca4556973 P 8c9a642ca63546dda148f11f2974bf01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c9a642ca63546dda148f11f2974bf01 (127.5.158.67). New cstate: current_term: 1 leader_uuid: "8c9a642ca63546dda148f11f2974bf01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c9a642ca63546dda148f11f2974bf01" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 43091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d36767c07fe04ceab87767a829be1a28" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 44029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f41581887534ad29bb86fbeee06df42" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 38887 } health_report { overall_health: UNKNOWN } } }
I20260925 07:27:51.496477 19027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:51.493266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47054 (local address 127.5.158.67:43091)
0925 07:27:51.493322 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:51.493324 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:51.493332 (+ 8us) server_negotiation.cc:407] Connection header received
0925 07:27:51.493363 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:51.493363 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:51.493381 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:51.493451 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 07:27:51.493590 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:51.494336 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:51.495431 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:51.495716 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:51.495777 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:51.495982 (+ 205us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 07:27:51.496089 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 07:27:51.496169 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:51.496393 (+ 224us) server_negotiation.cc:299] Negotiation successful
0925 07:27:51.496412 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260925 07:27:51.498513 18816 raft_consensus.cc:1261] T b92c4fa2a41a4bebaab5fd9ca4556973 P 2f41581887534ad29bb86fbeee06df42 [term 1 FOLLOWER]: Refusing update from remote peer 8c9a642ca63546dda148f11f2974bf01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:27:51.498811 19037 consensus_queue.cc:1035] T b92c4fa2a41a4bebaab5fd9ca4556973 P 8c9a642ca63546dda148f11f2974bf01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f41581887534ad29bb86fbeee06df42" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:27:51.498874 18682 raft_consensus.cc:1261] T b92c4fa2a41a4bebaab5fd9ca4556973 P d36767c07fe04ceab87767a829be1a28 [term 1 FOLLOWER]: Refusing update from remote peer 8c9a642ca63546dda148f11f2974bf01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:27:51.499161 19037 consensus_queue.cc:1035] T b92c4fa2a41a4bebaab5fd9ca4556973 P 8c9a642ca63546dda148f11f2974bf01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36767c07fe04ceab87767a829be1a28" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 44029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:27:51.502952 19046 mvcc.cc:205] Tried to move back new op lower bound from 7333155928055873536 to 7333155927954903040. Current Snapshot: MvccSnapshot[applied={T|T < 7333155928055873536 or (T in {7333155928055873536})}]
I20260925 07:27:51.503800 18530 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47668:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 07:27:51.503890 18530 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47668:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 07:27:51.504004 19047 mvcc.cc:205] Tried to move back new op lower bound from 7333155928055873536 to 7333155927954903040. Current Snapshot: MvccSnapshot[applied={T|T < 7333155928055873536 or (T in {7333155928055873536})}]
I20260925 07:27:51.504638 18530 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bbb0f321adbd492ab6df0b3d69b8bc9d: Sending DeleteTablet for 3 replicas of tablet b92c4fa2a41a4bebaab5fd9ca4556973
I20260925 07:27:51.505054 18662 tablet_service.cc:1514] Processing DeleteTablet for tablet b92c4fa2a41a4bebaab5fd9ca4556973 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:27:51 UTC) from {username='kudu', principal='kudu/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:33414
I20260925 07:27:51.505148 18930 tablet_service.cc:1514] Processing DeleteTablet for tablet b92c4fa2a41a4bebaab5fd9ca4556973 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:27:51 UTC) from {username='kudu', principal='kudu/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:47052
I20260925 07:27:51.505278 18782 tablet_service.cc:1514] Processing DeleteTablet for tablet b92c4fa2a41a4bebaab5fd9ca4556973 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:27:51 UTC) from {username='kudu', principal='kudu/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:50502
I20260925 07:27:51.505600 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 18568
I20260925 07:27:51.506131 19057 tablet_replica.cc:330] T b92c4fa2a41a4bebaab5fd9ca4556973 P 2f41581887534ad29bb86fbeee06df42: stopping tablet replica
I20260925 07:27:51.506242 19057 raft_consensus.cc:2229] T b92c4fa2a41a4bebaab5fd9ca4556973 P 2f41581887534ad29bb86fbeee06df42 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 07:27:51.506353 19057 raft_consensus.cc:2258] T b92c4fa2a41a4bebaab5fd9ca4556973 P 2f41581887534ad29bb86fbeee06df42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:27:51.506737 19057 ts_tablet_manager.cc:1905] T b92c4fa2a41a4bebaab5fd9ca4556973 P 2f41581887534ad29bb86fbeee06df42: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:27:51.508685 19057 ts_tablet_manager.cc:1918] T b92c4fa2a41a4bebaab5fd9ca4556973 P 2f41581887534ad29bb86fbeee06df42: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 07:27:51.508762 19057 log.cc:1196] T b92c4fa2a41a4bebaab5fd9ca4556973 P 2f41581887534ad29bb86fbeee06df42: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790321110323796-5753-0/minicluster-data/ts-1/wal/wals/b92c4fa2a41a4bebaab5fd9ca4556973
I20260925 07:27:51.509045 19057 ts_tablet_manager.cc:1939] T b92c4fa2a41a4bebaab5fd9ca4556973 P 2f41581887534ad29bb86fbeee06df42: Deleting consensus metadata
I20260925 07:27:51.509460 18515 catalog_manager.cc:4841] TS 2f41581887534ad29bb86fbeee06df42 (127.5.158.66:38887): tablet b92c4fa2a41a4bebaab5fd9ca4556973 (table test-table [id=8fa09b3b525748a2a2bc7fceedb7f5d0]) successfully deleted
W20260925 07:27:51.532086 18514 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.158.65:44029 (error 108)
I20260925 07:27:51.532618 19056 tablet_replica.cc:330] T b92c4fa2a41a4bebaab5fd9ca4556973 P 8c9a642ca63546dda148f11f2974bf01: stopping tablet replica
I20260925 07:27:51.532765 19056 raft_consensus.cc:2229] T b92c4fa2a41a4bebaab5fd9ca4556973 P 8c9a642ca63546dda148f11f2974bf01 [term 1 LEADER]: Raft consensus shutting down.
I20260925 07:27:51.532873 19056 raft_consensus.cc:2258] T b92c4fa2a41a4bebaab5fd9ca4556973 P 8c9a642ca63546dda148f11f2974bf01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:27:51.533046 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 18732
I20260925 07:27:51.538539 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 18865
I20260925 07:27:51.540562 19023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:51.540462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:44029 (local address 127.0.0.1:33426)
0925 07:27:51.540499 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:51.540536 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.65:44029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260925 07:27:51.540685 18514 catalog_manager.cc:4584] TS d36767c07fe04ceab87767a829be1a28 (127.5.158.65:44029): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b92c4fa2a41a4bebaab5fd9ca4556973: Network error: Client connection negotiation failed: client connection to 127.5.158.65:44029: connect: Connection refused (error 111)
I20260925 07:27:51.543429 19023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:51.543344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.67:43091 (local address 127.0.0.1:47060)
0925 07:27:51.543382 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:51.543399 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.67:43091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260925 07:27:51.543623 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 18497
I20260925 07:27:51.554785 5753 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260925 07:27:51.899821 5753 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260925 07:27:51.899946 5753 mini_ranger.cc:66] Stopping Ranger...
I20260925 07:27:52.244844 5753 mini_ranger.cc:68] Stopped Ranger
2026-09-25 07:27:52.245 UTC [17564] LOG: received smart shutdown request
2026-09-25 07:27:52.249 UTC [17564] LOG: background worker "logical replication launcher" (PID 17573) exited with exit code 1
2026-09-25 07:27:52.249 UTC [17568] LOG: shutting down
2026-09-25 07:27:52.395 UTC [17564] LOG: database system is shut down
2026-09-25T07:27:52Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (42042 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 07:27:52 dist-test-slave-1drq krb5kdc[19067](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 07:27:52 dist-test-slave-1drq krb5kdc[19067](info): set up 2 sockets
Sep 25 07:27:52 dist-test-slave-1drq krb5kdc[19067](info): commencing operation
krb5kdc: starting...
W20260925 07:27:53.457907 5753 mini_kdc.cc:121] Time spent starting KDC: real 0.948s user 0.003s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Sep 25 07:27:53 dist-test-slave-1drq krb5kdc[19067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321273, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T07:27:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T07:27:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
I20260925 07:27:53.562166 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:46431
--webserver_interface=127.5.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:36615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.158.126:46431,127.5.158.125:34443,127.5.158.124:44943
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:27:53.635743 19080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:27:53.635938 19080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:27:53.635984 19080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:27:53.637018 19080 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:27:53.637080 19080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:27:53.637130 19080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:27:53.637171 19080 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:27:53.637195 19080 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:27:53.638556 19080 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:36615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.158.126:46431,127.5.158.125:34443,127.5.158.124:44943
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:46431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19080
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:27:53.638793 19080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:27:53.639005 19080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:27:53.643487 19088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:27:53.643548 19086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:27:53.643498 19085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:27:53.643723 19080 server_base.cc:1034] running on GCE node
I20260925 07:27:53.644035 19080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:27:53.644313 19080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:27:53.645481 19080 hybrid_clock.cc:648] HybridClock initialized: now 1790321273645474 us; error 36 us; skew 500 ppm
Sep 25 07:27:53 dist-test-slave-1drq krb5kdc[19067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321273, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:27:53.652236 19080 init.cc:377] Logged in from keytab as kudu/127.5.158.126@KRBTEST.COM (short username kudu)
I20260925 07:27:53.652668 19080 webserver.cc:460] Webserver started at http://127.5.158.126:44103/ using document root <none> and password file <none>
I20260925 07:27:53.652873 19080 fs_manager.cc:359] Metadata directory not provided
I20260925 07:27:53.652930 19080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:27:53.653059 19080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:27:53.654012 19080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/minicluster-data/master-0/data/instance:
uuid: "0db28f4d6d194fea809f93c95a24e0af"
format_stamp: "Formatted at 2026-09-25 07:27:53 on dist-test-slave-1drq"
server_key: "a2a4705997fbf7cc7567c4f1862f7048"
server_key_iv: "bf9302f91844ff197e20bcce1a23566d"
server_key_version: "encryptionkey@0"
I20260925 07:27:53.654345 19080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/minicluster-data/master-0/wal/instance:
uuid: "0db28f4d6d194fea809f93c95a24e0af"
format_stamp: "Formatted at 2026-09-25 07:27:53 on dist-test-slave-1drq"
server_key: "a2a4705997fbf7cc7567c4f1862f7048"
server_key_iv: "bf9302f91844ff197e20bcce1a23566d"
server_key_version: "encryptionkey@0"
I20260925 07:27:53.655622 19080 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 07:27:53.656311 19095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:53.656468 19080 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 07:27:53.656564 19080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "0db28f4d6d194fea809f93c95a24e0af"
format_stamp: "Formatted at 2026-09-25 07:27:53 on dist-test-slave-1drq"
server_key: "a2a4705997fbf7cc7567c4f1862f7048"
server_key_iv: "bf9302f91844ff197e20bcce1a23566d"
server_key_version: "encryptionkey@0"
I20260925 07:27:53.656639 19080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:27:53.663599 19080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:27:53.665796 19080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:27:53.665942 19080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:27:53.669883 19080 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.126:46431
I20260925 07:27:53.669905 19147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.126:46431 every 8 connection(s)
I20260925 07:27:53.670205 19080 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:27:53.670903 19148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:27:53.672055 19148 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.158.126" port: 46431 } has no permanent_uuid. Determining permanent_uuid...
I20260925 07:27:53.675928 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 19080
I20260925 07:27:53.676059 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/minicluster-data/master-0/wal/instance
I20260925 07:27:53.676254 5753 external_mini_cluster.cc:1428] Setting key 888e5a73bdd1dde65f4deedbac055a62
Sep 25 07:27:53 dist-test-slave-1drq krb5kdc[19067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321273, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.126@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:27:53.681339 19151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:53.672783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44042 (local address 127.5.158.126:46431)
0925 07:27:53.672884 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:53.672887 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:53.672921 (+ 34us) server_negotiation.cc:407] Connection header received
0925 07:27:53.673400 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:53.673402 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:53.673420 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:53.673565 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:27:53.674094 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:53.674871 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:53.675708 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:53.675992 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:53.676103 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:53.678261 (+ 2158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:27:53.678266 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:27:53.678267 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:27:53.678272 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:27:53.679617 (+ 1345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:53.680194 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:53.680196 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:53.680196 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:53.680267 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:53.680545 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:53.680545 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:53.680546 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:53.680719 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:27:53.680774 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:53.681028 (+ 254us) server_negotiation.cc:299] Negotiation successful
0925 07:27:53.681186 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75,"thread_start_us":34,"threads_started":1}
I20260925 07:27:53.681505 19150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:53.672654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:46431 (local address 127.0.0.1:44042)
0925 07:27:53.672795 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:53.672843 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0925 07:27:53.673258 (+ 415us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:27:53.673570 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:27:53.673572 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:27:53.673839 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:27:53.673988 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:53.673992 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:53.675001 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:53.675001 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:53.675611 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:53.675613 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:53.676086 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:53.676088 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:53.676166 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:53.676583 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:27:53.676590 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:27:53.678174 (+ 1584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:27:53.679737 (+ 1563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:27:53.679763 (+ 26us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:27:53.679765 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:27:53.680067 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:27:53.680336 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:27:53.680337 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:27:53.680337 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:27:53.680465 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:27:53.680792 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:27:53.680800 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:27:53.680881 (+ 81us) client_negotiation.cc:769] Sending connection context
0925 07:27:53.680998 (+ 117us) client_negotiation.cc:240] Negotiation successful
0925 07:27:53.681309 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"mutex_wait_us":142,"thread_start_us":62,"threads_started":1}
I20260925 07:27:53.682219 19148 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.158.125" port: 34443 } has no permanent_uuid. Determining permanent_uuid...
I20260925 07:27:53.682564 19150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:53.682427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.125:34443 (local address 127.0.0.1:33000)
0925 07:27:53.682481 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:53.682536 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.125:34443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260925 07:27:53.682711 19096 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.158.125:34443: connect: Connection refused (error 111)
I20260925 07:27:53.683141 19150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:53.683052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.125:34443 (local address 127.0.0.1:33010)
0925 07:27:53.683102 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:53.683125 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.125:34443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260925 07:27:53.683284 19148 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.158.125:34443: Network error: Client connection negotiation failed: client connection to 127.5.158.125:34443: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.5.158.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.125.keytab.
Entry for principal kudu/127.5.158.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.125.keytab.
I20260925 07:27:53.700166 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.125:34443
--webserver_interface=127.5.158.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.125.keytab
--principal=kudu/127.5.158.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:36615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.158.126:46431,127.5.158.125:34443,127.5.158.124:44943
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 07:27:53.732585 19148 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.158.125" port: 34443 } attempt: 1
I20260925 07:27:53.733073 19150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:53.732935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.125:34443 (local address 127.0.0.1:33026)
0925 07:27:53.733018 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:53.733051 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.125:34443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260925 07:27:53.733458 19150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:53.733363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.125:34443 (local address 127.0.0.1:33040)
0925 07:27:53.733413 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:53.733440 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.125:34443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260925 07:27:53.733561 19148 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.158.125:34443: Network error: Client connection negotiation failed: client connection to 127.5.158.125:34443: connect: Connection refused (error 111)
W20260925 07:27:53.774978 19155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:27:53.775239 19155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:27:53.775287 19155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:27:53.777269 19155 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:27:53.777323 19155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:27:53.777357 19155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:27:53.777387 19155 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:27:53.777412 19155 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:27:53.779438 19155 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:36615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.158.126:46431,127.5.158.125:34443,127.5.158.124:44943
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.125.keytab
--principal=kudu/127.5.158.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.125:34443
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.158.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19155
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:27:53.779686 19155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:27:53.779902 19155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:27:53.785421 19161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:27:53.785408 19160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:27:53.785403 19155 server_base.cc:1034] running on GCE node
W20260925 07:27:53.785398 19163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:27:53.785823 19155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:27:53.786084 19155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:27:53.787248 19155 hybrid_clock.cc:648] HybridClock initialized: now 1790321273787232 us; error 29 us; skew 500 ppm
Sep 25 07:27:53 dist-test-slave-1drq krb5kdc[19067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321273, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:27:53.789484 19155 init.cc:377] Logged in from keytab as kudu/127.5.158.125@KRBTEST.COM (short username kudu)
I20260925 07:27:53.789878 19155 webserver.cc:460] Webserver started at http://127.5.158.125:38937/ using document root <none> and password file <none>
I20260925 07:27:53.790088 19155 fs_manager.cc:359] Metadata directory not provided
I20260925 07:27:53.790153 19155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:27:53.790277 19155 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:27:53.791639 19155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/minicluster-data/master-1/data/instance:
uuid: "8425ada9c46042d9bcdf232b878862ec"
format_stamp: "Formatted at 2026-09-25 07:27:53 on dist-test-slave-1drq"
server_key: "c41b8bc361d608b8ea74863e98dd6471"
server_key_iv: "64e0efd0f9b902e1fe3abcbdf30f765d"
server_key_version: "encryptionkey@0"
I20260925 07:27:53.792057 19155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/minicluster-data/master-1/wal/instance:
uuid: "8425ada9c46042d9bcdf232b878862ec"
format_stamp: "Formatted at 2026-09-25 07:27:53 on dist-test-slave-1drq"
server_key: "c41b8bc361d608b8ea74863e98dd6471"
server_key_iv: "64e0efd0f9b902e1fe3abcbdf30f765d"
server_key_version: "encryptionkey@0"
I20260925 07:27:53.793872 19155 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260925 07:27:53.794745 19170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:53.794895 19155 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:27:53.794977 19155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/minicluster-data/master-1/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/minicluster-data/master-1/wal
uuid: "8425ada9c46042d9bcdf232b878862ec"
format_stamp: "Formatted at 2026-09-25 07:27:53 on dist-test-slave-1drq"
server_key: "c41b8bc361d608b8ea74863e98dd6471"
server_key_iv: "64e0efd0f9b902e1fe3abcbdf30f765d"
server_key_version: "encryptionkey@0"
I20260925 07:27:53.795039 19155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:27:53.800376 19155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:27:53.801970 19148 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.158.125" port: 34443 } attempt: 2
I20260925 07:27:53.802310 19150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:53.802181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.125:34443 (local address 127.0.0.1:33044)
0925 07:27:53.802237 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:53.802275 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.125:34443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260925 07:27:53.802697 19150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:53.802571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.125:34443 (local address 127.0.0.1:33054)
0925 07:27:53.802645 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:53.802672 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.125:34443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260925 07:27:53.802728 19155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
W20260925 07:27:53.802788 19148 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.158.125:34443: Network error: Client connection negotiation failed: client connection to 127.5.158.125:34443: connect: Connection refused (error 111)
I20260925 07:27:53.802845 19155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:27:53.806859 19155 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.125:34443
I20260925 07:27:53.806941 19222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.125:34443 every 8 connection(s)
I20260925 07:27:53.807204 19155 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/minicluster-data/master-1/data/info.pb
I20260925 07:27:53.808096 19223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:27:53.810312 19223 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.158.126" port: 46431 } has no permanent_uuid. Determining permanent_uuid...
I20260925 07:27:53.814218 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 19155
I20260925 07:27:53.814352 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/minicluster-data/master-1/wal/instance
I20260925 07:27:53.814551 5753 external_mini_cluster.cc:1428] Setting key ee31a1e94bfc2292c05eac14b2f74e5b
Sep 25 07:27:53 dist-test-slave-1drq krb5kdc[19067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321273, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.125@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:27:53.821215 19151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:53.811009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44050 (local address 127.5.158.126:46431)
0925 07:27:53.811064 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:53.811066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:53.811300 (+ 234us) server_negotiation.cc:407] Connection header received
0925 07:27:53.812010 (+ 710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:53.812011 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:53.812021 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:53.812063 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:27:53.812860 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:53.813372 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:53.814997 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:53.815342 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:53.815415 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:53.817913 (+ 2498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:27:53.817915 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:27:53.817916 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:27:53.817924 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:27:53.819625 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:53.820109 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:53.820110 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:53.820111 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:53.820173 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:53.820453 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:53.820453 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:53.820454 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:53.820602 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:27:53.820681 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:53.820983 (+ 302us) server_negotiation.cc:299] Negotiation successful
0925 07:27:53.821072 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260925 07:27:53.821342 19225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:53.811004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:46431 (local address 127.0.0.1:44050)
0925 07:27:53.811178 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:53.811217 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 07:27:53.811909 (+ 692us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:27:53.812130 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:27:53.812133 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:27:53.812488 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:27:53.812774 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:53.812779 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:53.813447 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:53.813448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:53.814900 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:53.814902 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:53.815433 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:53.815434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:53.815603 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:53.816106 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:27:53.816115 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:27:53.817812 (+ 1697us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:27:53.819724 (+ 1912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:27:53.819731 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:27:53.819733 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:27:53.820023 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:27:53.820245 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:27:53.820245 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:27:53.820245 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:27:53.820380 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:27:53.820679 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:27:53.820686 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:27:53.820843 (+ 157us) client_negotiation.cc:769] Sending connection context
0925 07:27:53.820955 (+ 112us) client_negotiation.cc:240] Negotiation successful
0925 07:27:53.821121 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":83,"threads_started":1}
WARNING: no policy specified for kudu/127.5.158.124@KRBTEST.COM; defaulting to no policy
I20260925 07:27:53.822032 19223 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.158.125" port: 34443 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.124@KRBTEST.COM" created.
Sep 25 07:27:53 dist-test-slave-1drq krb5kdc[19067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321273, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.125@KRBTEST.COM for kudu/127.5.158.125@KRBTEST.COM
I20260925 07:27:53.830015 19225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:53.822259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.125:34443 (local address 127.0.0.1:33068)
0925 07:27:53.822303 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:53.822311 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 07:27:53.822343 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:27:53.822593 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:27:53.822594 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:27:53.822729 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:27:53.822764 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:53.822765 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:53.823628 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:53.823629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:53.824268 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:53.824270 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:53.824907 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:53.824908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:53.824983 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:53.825381 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:27:53.825386 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:27:53.826605 (+ 1219us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:27:53.828860 (+ 2255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:27:53.828861 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:27:53.828861 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:27:53.829108 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:27:53.829344 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:27:53.829345 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:27:53.829345 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:27:53.829411 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:27:53.829720 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:27:53.829721 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:27:53.829768 (+ 47us) client_negotiation.cc:769] Sending connection context
0925 07:27:53.829841 (+ 73us) client_negotiation.cc:240] Negotiation successful
0925 07:27:53.829930 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260925 07:27:53.830389 19227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:53.822307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33068 (local address 127.5.158.125:34443)
0925 07:27:53.822451 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:53.822454 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:53.822484 (+ 30us) server_negotiation.cc:407] Connection header received
0925 07:27:53.822505 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:53.822507 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:53.822516 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:53.822594 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:27:53.822833 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:53.823531 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:53.824357 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:53.824826 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:53.824910 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:53.826693 (+ 1783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:27:53.826700 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:27:53.826701 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:27:53.826707 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:27:53.828767 (+ 2060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:53.829194 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:53.829197 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:53.829197 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:53.829260 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:53.829476 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:53.829477 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:53.829478 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:53.829638 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:27:53.829725 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:53.829885 (+ 160us) server_negotiation.cc:299] Negotiation successful
0925 07:27:53.830010 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":114,"thread_start_us":70,"threads_started":1}
I20260925 07:27:53.830881 19223 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.158.124" port: 44943 } has no permanent_uuid. Determining permanent_uuid...
I20260925 07:27:53.831193 19225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:53.831069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.124:44943 (local address 127.0.0.1:54282)
0925 07:27:53.831121 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:53.831166 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.124:44943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260925 07:27:53.831310 19171 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.158.124:44943: connect: Connection refused (error 111)
I20260925 07:27:53.831746 19225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:53.831632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.124:44943 (local address 127.0.0.1:54298)
0925 07:27:53.831679 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:53.831708 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.124:44943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260925 07:27:53.831881 19223 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.158.124:44943: Network error: Client connection negotiation failed: client connection to 127.5.158.124:44943: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.124.keytab.
Entry for principal kudu/127.5.158.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.124.keytab.
I20260925 07:27:53.839334 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.124:44943
--webserver_interface=127.5.158.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.124.keytab
--principal=kudu/127.5.158.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:36615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.158.126:46431,127.5.158.125:34443,127.5.158.124:44943
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 07:27:53.881192 19223 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.158.124" port: 44943 } attempt: 1
I20260925 07:27:53.881589 19225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:53.881440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.124:44943 (local address 127.0.0.1:54304)
0925 07:27:53.881529 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:53.881563 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.124:44943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260925 07:27:53.881880 19225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:53.881816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.124:44943 (local address 127.0.0.1:54314)
0925 07:27:53.881841 (+ 25us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:53.881864 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.124:44943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260925 07:27:53.881987 19223 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.158.124:44943: Network error: Client connection negotiation failed: client connection to 127.5.158.124:44943: connect: Connection refused (error 111)
I20260925 07:27:53.893913 19148 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.158.125" port: 34443 } attempt: 3
Sep 25 07:27:53 dist-test-slave-1drq krb5kdc[19067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321273, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.126@KRBTEST.COM for kudu/127.5.158.125@KRBTEST.COM
I20260925 07:27:53.900563 19227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:53.894273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33070 (local address 127.5.158.125:34443)
0925 07:27:53.894328 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:53.894330 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:53.894339 (+ 9us) server_negotiation.cc:407] Connection header received
0925 07:27:53.894363 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:53.894364 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:53.894375 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:53.894458 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:27:53.894727 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:53.895454 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:53.896027 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:53.896306 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:53.896368 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:53.898322 (+ 1954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:27:53.898323 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:27:53.898323 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:27:53.898328 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:27:53.899468 (+ 1140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:53.899813 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:53.899814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:53.899814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:53.899874 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:53.900086 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:53.900087 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:53.900087 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:53.900211 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:27:53.900259 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:53.900456 (+ 197us) server_negotiation.cc:299] Negotiation successful
0925 07:27:53.900511 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260925 07:27:53.900645 19150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:53.894174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.125:34443 (local address 127.0.0.1:33070)
0925 07:27:53.894225 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:53.894238 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 07:27:53.894287 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:27:53.894461 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:27:53.894462 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:27:53.894594 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:27:53.894654 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:53.894655 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:53.895532 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:53.895533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:53.895959 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:53.895960 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:53.896371 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:53.896371 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:53.896425 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:53.896830 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:27:53.896833 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:27:53.898249 (+ 1416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:27:53.899540 (+ 1291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:27:53.899542 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:27:53.899542 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:27:53.899742 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:27:53.899931 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:27:53.899931 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:27:53.899932 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:27:53.899997 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:27:53.900279 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:27:53.900279 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:27:53.900355 (+ 76us) client_negotiation.cc:769] Sending connection context
0925 07:27:53.900444 (+ 89us) client_negotiation.cc:240] Negotiation successful
0925 07:27:53.900550 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260925 07:27:53.901250 19148 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.158.124" port: 44943 } has no permanent_uuid. Determining permanent_uuid...
I20260925 07:27:53.901603 19150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:53.901499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.124:44943 (local address 127.0.0.1:54328)
0925 07:27:53.901548 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:53.901576 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.124:44943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260925 07:27:53.901912 19150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:53.901821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.124:44943 (local address 127.0.0.1:54334)
0925 07:27:53.901858 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:53.901885 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.124:44943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260925 07:27:53.902017 19148 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.158.124:44943: Network error: Client connection negotiation failed: client connection to 127.5.158.124:44943: connect: Connection refused (error 111)
W20260925 07:27:53.915262 19230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:27:53.915414 19230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:27:53.915433 19230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:27:53.916419 19230 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:27:53.916452 19230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:27:53.916465 19230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:27:53.916477 19230 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:27:53.916525 19230 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:27:53.917863 19230 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:36615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.158.126:46431,127.5.158.125:34443,127.5.158.124:44943
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.124.keytab
--principal=kudu/127.5.158.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.124:44943
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.158.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19230
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:27:53.918071 19230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:27:53.918253 19230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:27:53.922648 19235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:27:53.922677 19230 server_base.cc:1034] running on GCE node
W20260925 07:27:53.922652 19238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:27:53.922662 19236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:27:53.923058 19230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:27:53.923300 19230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:27:53.924446 19230 hybrid_clock.cc:648] HybridClock initialized: now 1790321273924431 us; error 29 us; skew 500 ppm
Sep 25 07:27:53 dist-test-slave-1drq krb5kdc[19067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321273, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:27:53.926609 19230 init.cc:377] Logged in from keytab as kudu/127.5.158.124@KRBTEST.COM (short username kudu)
I20260925 07:27:53.926986 19230 webserver.cc:460] Webserver started at http://127.5.158.124:40707/ using document root <none> and password file <none>
I20260925 07:27:53.927153 19230 fs_manager.cc:359] Metadata directory not provided
I20260925 07:27:53.927189 19230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:27:53.927264 19230 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:27:53.928093 19230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/minicluster-data/master-2/data/instance:
uuid: "2796bef7589b4d27b8de93c3f83683e4"
format_stamp: "Formatted at 2026-09-25 07:27:53 on dist-test-slave-1drq"
server_key: "003d56d5c36b20a7c1767826d2f2ebe5"
server_key_iv: "acaac36d475863f567529e54d4c974f2"
server_key_version: "encryptionkey@0"
I20260925 07:27:53.928349 19230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/minicluster-data/master-2/wal/instance:
uuid: "2796bef7589b4d27b8de93c3f83683e4"
format_stamp: "Formatted at 2026-09-25 07:27:53 on dist-test-slave-1drq"
server_key: "003d56d5c36b20a7c1767826d2f2ebe5"
server_key_iv: "acaac36d475863f567529e54d4c974f2"
server_key_version: "encryptionkey@0"
I20260925 07:27:53.929536 19230 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 07:27:53.930099 19245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:53.930250 19230 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:27:53.930337 19230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/minicluster-data/master-2/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/minicluster-data/master-2/wal
uuid: "2796bef7589b4d27b8de93c3f83683e4"
format_stamp: "Formatted at 2026-09-25 07:27:53 on dist-test-slave-1drq"
server_key: "003d56d5c36b20a7c1767826d2f2ebe5"
server_key_iv: "acaac36d475863f567529e54d4c974f2"
server_key_version: "encryptionkey@0"
I20260925 07:27:53.930408 19230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:27:53.933156 19148 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.158.124" port: 44943 } attempt: 1
I20260925 07:27:53.933470 19150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:53.933369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.124:44943 (local address 127.0.0.1:54336)
0925 07:27:53.933415 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:53.933445 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.124:44943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260925 07:27:53.933810 19150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:53.933701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.124:44943 (local address 127.0.0.1:54344)
0925 07:27:53.933761 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:53.933791 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.124:44943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260925 07:27:53.933916 19148 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.158.124:44943: Network error: Client connection negotiation failed: client connection to 127.5.158.124:44943: connect: Connection refused (error 111)
I20260925 07:27:53.943782 19230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:27:53.945997 19230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:27:53.946152 19230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:27:53.949802 19230 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.124:44943
I20260925 07:27:53.949836 19297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.124:44943 every 8 connection(s)
I20260925 07:27:53.950138 19223 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.158.124" port: 44943 } attempt: 2
I20260925 07:27:53.950260 19230 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/minicluster-data/master-2/data/info.pb
I20260925 07:27:53.951036 19298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:27:53.952337 19298 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.158.126" port: 46431 } has no permanent_uuid. Determining permanent_uuid...
I20260925 07:27:53.953504 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 19230
I20260925 07:27:53.953647 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790321110323796-5753-0/minicluster-data/master-2/wal/instance
I20260925 07:27:53.953873 5753 external_mini_cluster.cc:1428] Setting key 2a177cffe9410a8deb5c520cf8d8c1cf
Sep 25 07:27:53 dist-test-slave-1drq krb5kdc[19067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321273, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.125@KRBTEST.COM for kudu/127.5.158.124@KRBTEST.COM
Sep 25 07:27:53 dist-test-slave-1drq krb5kdc[19067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321273, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.124@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
Sep 25 07:27:53 dist-test-slave-1drq krb5kdc[19067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321273, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.158.124@KRBTEST.COM
I20260925 07:27:53.959537 19225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:53.950399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.124:44943 (local address 127.0.0.1:54346)
0925 07:27:53.950435 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:53.950467 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0925 07:27:53.950512 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:27:53.951410 (+ 898us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:27:53.951411 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:27:53.951576 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:27:53.951611 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:53.951612 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:53.952553 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:53.952555 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:53.953292 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:53.953295 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:53.953970 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:53.953971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:53.954027 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:53.954384 (+ 357us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:27:53.954388 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:27:53.955787 (+ 1399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:27:53.958240 (+ 2453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:27:53.958241 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:27:53.958241 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:27:53.958410 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:27:53.958724 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:27:53.958725 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:27:53.958726 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:27:53.958797 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:27:53.959256 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:27:53.959257 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:27:53.959301 (+ 44us) client_negotiation.cc:769] Sending connection context
0925 07:27:53.959348 (+ 47us) client_negotiation.cc:240] Negotiation successful
0925 07:27:53.959438 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260925 07:27:53.960137 19300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:53.950564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54346 (local address 127.5.158.124:44943)
0925 07:27:53.950729 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:53.950741 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:53.950762 (+ 21us) server_negotiation.cc:407] Connection header received
0925 07:27:53.951178 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:53.951185 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:53.951245 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:53.951428 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:27:53.951704 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:53.952470 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:53.953356 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:53.953860 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:53.953981 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:53.955864 (+ 1883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:27:53.955868 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:27:53.955868 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:27:53.955878 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:27:53.958158 (+ 2280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:53.958500 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:53.958503 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:53.958503 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:53.958627 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:53.958868 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:53.958869 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:53.958870 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:53.959163 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:27:53.959718 (+ 555us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:53.959798 (+ 80us) server_negotiation.cc:299] Negotiation successful
0925 07:27:53.959906 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":95,"threads_started":1}
I20260925 07:27:53.961795 19223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8425ada9c46042d9bcdf232b878862ec: Bootstrap starting.
I20260925 07:27:53.962126 19301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:53.952861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:46431 (local address 127.0.0.1:44064)
0925 07:27:53.953023 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:53.953048 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0925 07:27:53.953144 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:27:53.953270 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:27:53.953271 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:27:53.953576 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:27:53.953684 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:53.953687 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:53.954675 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:53.954675 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:53.955578 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:53.955581 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:53.956052 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:53.956053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:53.956125 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:53.956509 (+ 384us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:27:53.956514 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:27:53.958178 (+ 1664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:27:53.960166 (+ 1988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:27:53.960171 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:27:53.960173 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:27:53.961274 (+ 1101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:27:53.961484 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:27:53.961484 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:27:53.961485 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:27:53.961545 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:27:53.961813 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:27:53.961819 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:27:53.961865 (+ 46us) client_negotiation.cc:769] Sending connection context
0925 07:27:53.961933 (+ 68us) client_negotiation.cc:240] Negotiation successful
0925 07:27:53.962018 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":65,"threads_started":1}
I20260925 07:27:53.962146 19151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:53.952910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44064 (local address 127.5.158.126:46431)
0925 07:27:53.952956 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:53.952958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:53.953123 (+ 165us) server_negotiation.cc:407] Connection header received
0925 07:27:53.953191 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:53.953192 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:53.953201 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:53.953258 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:27:53.953820 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:53.954544 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:53.955665 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:53.955935 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:53.955971 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:53.958285 (+ 2314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:27:53.958287 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:27:53.958287 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:27:53.958294 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:27:53.960069 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:53.961355 (+ 1286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:53.961356 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:53.961357 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:53.961413 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:53.961623 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:53.961624 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:53.961624 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:53.961747 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:27:53.961810 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:53.961983 (+ 173us) server_negotiation.cc:299] Negotiation successful
0925 07:27:53.962053 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260925 07:27:53.962745 19223 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8425ada9c46042d9bcdf232b878862ec: Neither blocks nor log segments found. Creating new log.
I20260925 07:27:53.962766 19298 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.158.125" port: 34443 } has no permanent_uuid. Determining permanent_uuid...
I20260925 07:27:53.962991 19302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:53.954152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54356 (local address 127.5.158.124:44943)
0925 07:27:53.954290 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:53.954292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:53.954631 (+ 339us) server_negotiation.cc:407] Connection header received
0925 07:27:53.954677 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:53.954678 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:53.954688 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:53.954759 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:27:53.955352 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:53.955851 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:53.956801 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:53.956992 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:53.957058 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:53.959721 (+ 2663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:27:53.959722 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:27:53.959722 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:27:53.959729 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:27:53.961855 (+ 2126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:53.962222 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:53.962223 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:53.962223 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:53.962272 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:53.962465 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:53.962466 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:53.962466 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:53.962570 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:27:53.962635 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:53.962809 (+ 174us) server_negotiation.cc:299] Negotiation successful
0925 07:27:53.962874 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":91,"threads_started":1}
I20260925 07:27:53.963052 19223 log.cc:824] T 00000000000000000000000000000000 P 8425ada9c46042d9bcdf232b878862ec: Log is configured to *not* fsync() on all Append() calls
I20260925 07:27:53.963840 19223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8425ada9c46042d9bcdf232b878862ec: No bootstrap required, opened a new log
I20260925 07:27:53.965759 19223 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8425ada9c46042d9bcdf232b878862ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0db28f4d6d194fea809f93c95a24e0af" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 46431 } } peers { permanent_uuid: "8425ada9c46042d9bcdf232b878862ec" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 34443 } } peers { permanent_uuid: "2796bef7589b4d27b8de93c3f83683e4" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 44943 } }
I20260925 07:27:53.965886 19223 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8425ada9c46042d9bcdf232b878862ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:27:53.965930 19223 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8425ada9c46042d9bcdf232b878862ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8425ada9c46042d9bcdf232b878862ec, State: Initialized, Role: FOLLOWER
I20260925 07:27:53.966053 19223 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8425ada9c46042d9bcdf232b878862ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0db28f4d6d194fea809f93c95a24e0af" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 46431 } } peers { permanent_uuid: "8425ada9c46042d9bcdf232b878862ec" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 34443 } } peers { permanent_uuid: "2796bef7589b4d27b8de93c3f83683e4" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 44943 } }
I20260925 07:27:53.966320 19223 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8425ada9c46042d9bcdf232b878862ec [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:27:53.966289 19305 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8425ada9c46042d9bcdf232b878862ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0db28f4d6d194fea809f93c95a24e0af" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 46431 } } peers { permanent_uuid: "8425ada9c46042d9bcdf232b878862ec" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 34443 } } peers { permanent_uuid: "2796bef7589b4d27b8de93c3f83683e4" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 44943 } } }
I20260925 07:27:53.966370 19305 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8425ada9c46042d9bcdf232b878862ec [sys.catalog]: This master's current role is: FOLLOWER
Sep 25 07:27:53 dist-test-slave-1drq krb5kdc[19067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321273, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.124@KRBTEST.COM for kudu/127.5.158.125@KRBTEST.COM
Sep 25 07:27:53 dist-test-slave-1drq krb5kdc[19067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321273, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
W20260925 07:27:53.968956 19316 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8425ada9c46042d9bcdf232b878862ec: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 07:27:53.969033 19316 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 07:27:53.970577 19301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:53.963039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.125:34443 (local address 127.0.0.1:33076)
0925 07:27:53.963272 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:53.963283 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 07:27:53.963335 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:27:53.963543 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:27:53.963544 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:27:53.963675 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:27:53.963729 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:53.963731 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:53.964601 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:53.964601 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:53.965319 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:53.965321 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:53.965760 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:53.965761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:53.965822 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:53.966201 (+ 379us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:27:53.966206 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:27:53.968006 (+ 1800us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:27:53.969599 (+ 1593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:27:53.969600 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:27:53.969600 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:27:53.969795 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:27:53.969971 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:27:53.969971 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:27:53.969971 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:27:53.970022 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:27:53.970331 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:27:53.970332 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:27:53.970368 (+ 36us) client_negotiation.cc:769] Sending connection context
0925 07:27:53.970426 (+ 58us) client_negotiation.cc:240] Negotiation successful
0925 07:27:53.970503 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220}
I20260925 07:27:53.970830 19227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:53.963112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33076 (local address 127.5.158.125:34443)
0925 07:27:53.963155 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:53.963157 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:53.963455 (+ 298us) server_negotiation.cc:407] Connection header received
0925 07:27:53.963475 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:53.963476 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:53.963484 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:53.963537 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:27:53.963850 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:53.964502 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:53.965406 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:53.965657 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:53.965693 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:53.968078 (+ 2385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:27:53.968079 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:27:53.968079 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:27:53.968086 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:27:53.969514 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:53.969864 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:53.969865 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:53.969865 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:53.969912 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:53.970099 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:53.970100 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:53.970100 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:53.970231 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:27:53.970305 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:53.970688 (+ 383us) server_negotiation.cc:299] Negotiation successful
0925 07:27:53.970756 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 07:27:53.971416 19151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:53.963479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44076 (local address 127.5.158.126:46431)
0925 07:27:53.963516 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:53.963518 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:53.963633 (+ 115us) server_negotiation.cc:407] Connection header received
0925 07:27:53.963698 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:53.963699 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:53.963707 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:53.963778 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:27:53.964280 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:53.964704 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:53.965408 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:53.965594 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:53.965668 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:53.968203 (+ 2535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:27:53.968204 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:27:53.968204 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:27:53.968210 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:27:53.969778 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:53.970204 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:53.970205 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:53.970205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:53.970265 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:53.970473 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:53.970474 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:53.970474 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:53.970584 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:27:53.970719 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:53.971266 (+ 547us) server_negotiation.cc:299] Negotiation successful
0925 07:27:53.971333 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 07:27:53.968648 19155 master_runner.cc:186] Error getting master registration for 127.5.158.126:46431: OK, instance_id { permanent_uuid: "0db28f4d6d194fea809f93c95a24e0af" instance_seqno: 1790321273668628 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 07:27:53.971982 19155 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 07:27:53.972368 19298 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.158.124" port: 44943 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 07:27:53 dist-test-slave-1drq krb5kdc[19067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321273, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.124@KRBTEST.COM for kudu/127.5.158.124@KRBTEST.COM
Sep 25 07:27:53 dist-test-slave-1drq krb5kdc[19067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321273, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.158.125@KRBTEST.COM
I20260925 07:27:53.979171 19301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:53.972560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.124:44943 (local address 127.0.0.1:54366)
0925 07:27:53.972589 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:53.972598 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 07:27:53.972636 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:27:53.972763 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:27:53.972764 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:27:53.972879 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:27:53.972898 (+ 19us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:53.972899 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:53.973516 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:53.973517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:53.974006 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:53.974007 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:53.974500 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:53.974501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:53.974554 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:53.974924 (+ 370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:27:53.974929 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:27:53.976120 (+ 1191us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:27:53.977740 (+ 1620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:27:53.977741 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:27:53.977741 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:27:53.977954 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:27:53.978188 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:27:53.978188 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:27:53.978189 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:27:53.978249 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:27:53.978862 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:27:53.978864 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:27:53.978916 (+ 52us) client_negotiation.cc:769] Sending connection context
0925 07:27:53.979002 (+ 86us) client_negotiation.cc:240] Negotiation successful
0925 07:27:53.979090 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260925 07:27:53.979458 19302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:53.972616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54366 (local address 127.5.158.124:44943)
0925 07:27:53.972643 (+ 27us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:53.972644 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:53.972651 (+ 7us) server_negotiation.cc:407] Connection header received
0925 07:27:53.972687 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:53.972688 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:53.972695 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:53.972760 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:27:53.972964 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:53.973428 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:53.974108 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:53.974422 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:53.974491 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:53.976200 (+ 1709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:27:53.976201 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:27:53.976201 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:27:53.976207 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:27:53.977676 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:53.978046 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:53.978047 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:53.978047 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:53.978122 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:53.978326 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:53.978327 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:53.978328 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:53.978768 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:27:53.978838 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:53.979316 (+ 478us) server_negotiation.cc:299] Negotiation successful
0925 07:27:53.979384 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260925 07:27:53.980571 19298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2796bef7589b4d27b8de93c3f83683e4: Bootstrap starting.
I20260925 07:27:53.980798 19227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:53.972501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33082 (local address 127.5.158.125:34443)
0925 07:27:53.972530 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:53.972532 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:53.972699 (+ 167us) server_negotiation.cc:407] Connection header received
0925 07:27:53.972716 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:53.972717 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:53.972724 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:53.972765 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:27:53.973319 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:53.973999 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:53.975006 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:53.975279 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:53.975344 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:53.978188 (+ 2844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:27:53.978189 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:27:53.978189 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:27:53.978194 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:27:53.979776 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:53.980085 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:53.980086 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:53.980086 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:53.980133 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:53.980346 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:53.980347 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:53.980348 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:53.980450 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:27:53.980601 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:53.980639 (+ 38us) server_negotiation.cc:299] Negotiation successful
0925 07:27:53.980708 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260925 07:27:53.981209 19298 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2796bef7589b4d27b8de93c3f83683e4: Neither blocks nor log segments found. Creating new log.
I20260925 07:27:53.981484 19298 log.cc:824] T 00000000000000000000000000000000 P 2796bef7589b4d27b8de93c3f83683e4: Log is configured to *not* fsync() on all Append() calls
I20260925 07:27:53.981999 5753 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 07:27:53.982213 19298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2796bef7589b4d27b8de93c3f83683e4: No bootstrap required, opened a new log
I20260925 07:27:53.983963 19298 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2796bef7589b4d27b8de93c3f83683e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0db28f4d6d194fea809f93c95a24e0af" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 46431 } } peers { permanent_uuid: "8425ada9c46042d9bcdf232b878862ec" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 34443 } } peers { permanent_uuid: "2796bef7589b4d27b8de93c3f83683e4" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 44943 } }
I20260925 07:27:53.984143 19298 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2796bef7589b4d27b8de93c3f83683e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:27:53.984191 19298 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2796bef7589b4d27b8de93c3f83683e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2796bef7589b4d27b8de93c3f83683e4, State: Initialized, Role: FOLLOWER
I20260925 07:27:53.984303 19298 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2796bef7589b4d27b8de93c3f83683e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0db28f4d6d194fea809f93c95a24e0af" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 46431 } } peers { permanent_uuid: "8425ada9c46042d9bcdf232b878862ec" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 34443 } } peers { permanent_uuid: "2796bef7589b4d27b8de93c3f83683e4" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 44943 } }
I20260925 07:27:53.984622 19298 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2796bef7589b4d27b8de93c3f83683e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:27:53.989584 19326 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2796bef7589b4d27b8de93c3f83683e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0db28f4d6d194fea809f93c95a24e0af" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 46431 } } peers { permanent_uuid: "8425ada9c46042d9bcdf232b878862ec" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 34443 } } peers { permanent_uuid: "2796bef7589b4d27b8de93c3f83683e4" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 44943 } } }
I20260925 07:27:53.990042 19326 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2796bef7589b4d27b8de93c3f83683e4 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 07:27:53.990828 19227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:53.983539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33086 (local address 127.5.158.125:34443)
0925 07:27:53.983574 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:53.983575 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:53.983581 (+ 6us) server_negotiation.cc:407] Connection header received
0925 07:27:53.983597 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:53.983597 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:53.983604 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:53.983653 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:27:53.984335 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:53.985034 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:53.986031 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:53.986331 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:53.986396 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:53.987748 (+ 1352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:27:53.987749 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:27:53.987750 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:27:53.987756 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:27:53.989755 (+ 1999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:53.990082 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:53.990083 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:53.990083 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:53.990126 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:53.990317 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:53.990318 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:53.990318 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:53.990423 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:27:53.990480 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:53.990682 (+ 202us) server_negotiation.cc:299] Negotiation successful
0925 07:27:53.990750 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 07:27:53.986996 19230 master_runner.cc:186] Error getting master registration for 127.5.158.126:46431: OK, instance_id { permanent_uuid: "0db28f4d6d194fea809f93c95a24e0af" instance_seqno: 1790321273668628 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 07:27:53.991777 19230 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 07:27:53.992218 19302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:53.983606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54372 (local address 127.5.158.124:44943)
0925 07:27:53.983634 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:53.983636 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:53.984143 (+ 507us) server_negotiation.cc:407] Connection header received
0925 07:27:53.984366 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:53.984367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:53.984375 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:53.984403 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:27:53.985307 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:53.985980 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:53.987066 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:53.987331 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:53.987368 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:53.989286 (+ 1918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:27:53.989287 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:27:53.989288 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:27:53.989293 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:27:53.990760 (+ 1467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:53.991156 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:53.991158 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:53.991158 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:53.991214 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:53.991445 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:53.991447 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:53.991447 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:53.991583 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:27:53.991670 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:53.992021 (+ 351us) server_negotiation.cc:299] Negotiation successful
0925 07:27:53.992102 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
W20260925 07:27:53.993067 19337 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2796bef7589b4d27b8de93c3f83683e4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 07:27:53.993191 19337 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 07:27:53.996165 19151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:53.983781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44090 (local address 127.5.158.126:46431)
0925 07:27:53.983832 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:53.983834 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:53.987725 (+ 3891us) server_negotiation.cc:407] Connection header received
0925 07:27:53.987826 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:53.987827 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:53.987836 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:53.987867 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:27:53.988445 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:53.989064 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:53.991852 (+ 2788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:53.992144 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:53.992190 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:53.993493 (+ 1303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:27:53.993494 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:27:53.993495 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:27:53.993501 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:27:53.995050 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:53.995418 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:53.995419 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:53.995419 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:53.995461 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:53.995625 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:53.995625 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:53.995625 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:53.995735 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:27:53.995822 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:53.996022 (+ 200us) server_negotiation.cc:299] Negotiation successful
0925 07:27:53.996087 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260925 07:27:54.009063 19148 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.158.124" port: 44943 } attempt: 2
Sep 25 07:27:54 dist-test-slave-1drq krb5kdc[19067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321273, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.126@KRBTEST.COM for kudu/127.5.158.124@KRBTEST.COM
I20260925 07:27:54.016204 19150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:54.009292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.124:44943 (local address 127.0.0.1:54374)
0925 07:27:54.009330 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:54.009339 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 07:27:54.009369 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:27:54.009521 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:27:54.009522 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:27:54.009668 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:27:54.009700 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:54.009702 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:54.010530 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:54.010531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:54.011260 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:54.011262 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:54.011723 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:54.011724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:54.011774 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:54.012129 (+ 355us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:27:54.012133 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:27:54.013457 (+ 1324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:27:54.015222 (+ 1765us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:27:54.015223 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:27:54.015224 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:27:54.015409 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:27:54.015599 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:27:54.015600 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:27:54.015600 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:27:54.015669 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:27:54.015920 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:27:54.015921 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:27:54.015960 (+ 39us) client_negotiation.cc:769] Sending connection context
0925 07:27:54.016036 (+ 76us) client_negotiation.cc:240] Negotiation successful
0925 07:27:54.016117 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260925 07:27:54.016222 19302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:54.009322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54374 (local address 127.5.158.124:44943)
0925 07:27:54.009352 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:54.009354 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:54.009361 (+ 7us) server_negotiation.cc:407] Connection header received
0925 07:27:54.009436 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:54.009437 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:54.009446 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:54.009487 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:27:54.009773 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:54.010450 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:54.011364 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:54.011652 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:54.011715 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:54.013547 (+ 1832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:27:54.013548 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:27:54.013548 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:27:54.013554 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:27:54.015124 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:54.015479 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:54.015480 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:54.015480 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:54.015526 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:54.015741 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:54.015742 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:54.015742 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:54.015853 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:27:54.015916 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:54.016069 (+ 153us) server_negotiation.cc:299] Negotiation successful
0925 07:27:54.016144 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260925 07:27:54.017752 19148 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0db28f4d6d194fea809f93c95a24e0af: Bootstrap starting.
I20260925 07:27:54.018378 19148 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0db28f4d6d194fea809f93c95a24e0af: Neither blocks nor log segments found. Creating new log.
I20260925 07:27:54.018649 19148 log.cc:824] T 00000000000000000000000000000000 P 0db28f4d6d194fea809f93c95a24e0af: Log is configured to *not* fsync() on all Append() calls
I20260925 07:27:54.019400 19148 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0db28f4d6d194fea809f93c95a24e0af: No bootstrap required, opened a new log
I20260925 07:27:54.021266 19148 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0db28f4d6d194fea809f93c95a24e0af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0db28f4d6d194fea809f93c95a24e0af" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 46431 } } peers { permanent_uuid: "8425ada9c46042d9bcdf232b878862ec" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 34443 } } peers { permanent_uuid: "2796bef7589b4d27b8de93c3f83683e4" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 44943 } }
I20260925 07:27:54.021389 19148 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0db28f4d6d194fea809f93c95a24e0af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:27:54.021421 19148 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0db28f4d6d194fea809f93c95a24e0af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0db28f4d6d194fea809f93c95a24e0af, State: Initialized, Role: FOLLOWER
I20260925 07:27:54.021524 19148 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0db28f4d6d194fea809f93c95a24e0af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0db28f4d6d194fea809f93c95a24e0af" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 46431 } } peers { permanent_uuid: "8425ada9c46042d9bcdf232b878862ec" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 34443 } } peers { permanent_uuid: "2796bef7589b4d27b8de93c3f83683e4" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 44943 } }
I20260925 07:27:54.021803 19148 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0db28f4d6d194fea809f93c95a24e0af [sys.catalog]: configured and running, proceeding with master startup.
W20260925 07:27:54.023442 19351 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0db28f4d6d194fea809f93c95a24e0af: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 07:27:54.023532 19351 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 07:27:54.025240 19080 master_runner.cc:263] No leader master found from master 0db28f4d6d194fea809f93c95a24e0af
I20260925 07:27:54.025290 19080 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 07:27:54.022960 19340 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0db28f4d6d194fea809f93c95a24e0af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0db28f4d6d194fea809f93c95a24e0af" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 46431 } } peers { permanent_uuid: "8425ada9c46042d9bcdf232b878862ec" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 34443 } } peers { permanent_uuid: "2796bef7589b4d27b8de93c3f83683e4" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 44943 } } }
I20260925 07:27:54.025334 19340 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0db28f4d6d194fea809f93c95a24e0af [sys.catalog]: This master's current role is: FOLLOWER
W20260925 07:27:54.053398 19176 tablet.cc:2369] T 00000000000000000000000000000000 P 8425ada9c46042d9bcdf232b878862ec: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 07:27:54.128576 19305 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8425ada9c46042d9bcdf232b878862ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 07:27:54.128687 19305 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8425ada9c46042d9bcdf232b878862ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0db28f4d6d194fea809f93c95a24e0af" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 46431 } } peers { permanent_uuid: "8425ada9c46042d9bcdf232b878862ec" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 34443 } } peers { permanent_uuid: "2796bef7589b4d27b8de93c3f83683e4" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 44943 } }
I20260925 07:27:54.128923 19305 leader_election.cc:288] T 00000000000000000000000000000000 P 8425ada9c46042d9bcdf232b878862ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0db28f4d6d194fea809f93c95a24e0af (127.5.158.126:46431), 2796bef7589b4d27b8de93c3f83683e4 (127.5.158.124:44943)
I20260925 07:27:54.129217 19272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8425ada9c46042d9bcdf232b878862ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2796bef7589b4d27b8de93c3f83683e4" is_pre_election: true
I20260925 07:27:54.129225 19122 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8425ada9c46042d9bcdf232b878862ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0db28f4d6d194fea809f93c95a24e0af" is_pre_election: true
I20260925 07:27:54.129345 19272 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2796bef7589b4d27b8de93c3f83683e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8425ada9c46042d9bcdf232b878862ec in term 0.
I20260925 07:27:54.129345 19122 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0db28f4d6d194fea809f93c95a24e0af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8425ada9c46042d9bcdf232b878862ec in term 0.
I20260925 07:27:54.129519 19171 leader_election.cc:302] T 00000000000000000000000000000000 P 8425ada9c46042d9bcdf232b878862ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2796bef7589b4d27b8de93c3f83683e4, 8425ada9c46042d9bcdf232b878862ec; no voters:
I20260925 07:27:54.129643 19305 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8425ada9c46042d9bcdf232b878862ec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 07:27:54.129688 19305 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8425ada9c46042d9bcdf232b878862ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 07:27:54.129712 19305 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8425ada9c46042d9bcdf232b878862ec [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:27:54.130330 19305 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8425ada9c46042d9bcdf232b878862ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0db28f4d6d194fea809f93c95a24e0af" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 46431 } } peers { permanent_uuid: "8425ada9c46042d9bcdf232b878862ec" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 34443 } } peers { permanent_uuid: "2796bef7589b4d27b8de93c3f83683e4" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 44943 } }
I20260925 07:27:54.130450 19305 leader_election.cc:288] T 00000000000000000000000000000000 P 8425ada9c46042d9bcdf232b878862ec [CANDIDATE]: Term 1 election: Requested vote from peers 0db28f4d6d194fea809f93c95a24e0af (127.5.158.126:46431), 2796bef7589b4d27b8de93c3f83683e4 (127.5.158.124:44943)
I20260925 07:27:54.130621 19122 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8425ada9c46042d9bcdf232b878862ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0db28f4d6d194fea809f93c95a24e0af"
I20260925 07:27:54.130609 19272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8425ada9c46042d9bcdf232b878862ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2796bef7589b4d27b8de93c3f83683e4"
I20260925 07:27:54.130677 19122 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0db28f4d6d194fea809f93c95a24e0af [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:27:54.130677 19272 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2796bef7589b4d27b8de93c3f83683e4 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:27:54.131428 19122 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0db28f4d6d194fea809f93c95a24e0af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8425ada9c46042d9bcdf232b878862ec in term 1.
I20260925 07:27:54.131428 19272 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2796bef7589b4d27b8de93c3f83683e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8425ada9c46042d9bcdf232b878862ec in term 1.
I20260925 07:27:54.131600 19171 leader_election.cc:302] T 00000000000000000000000000000000 P 8425ada9c46042d9bcdf232b878862ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2796bef7589b4d27b8de93c3f83683e4, 8425ada9c46042d9bcdf232b878862ec; no voters:
I20260925 07:27:54.131685 19305 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8425ada9c46042d9bcdf232b878862ec [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:27:54.131855 19305 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8425ada9c46042d9bcdf232b878862ec [term 1 LEADER]: Becoming Leader. State: Replica: 8425ada9c46042d9bcdf232b878862ec, State: Running, Role: LEADER
I20260925 07:27:54.131949 19305 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8425ada9c46042d9bcdf232b878862ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0db28f4d6d194fea809f93c95a24e0af" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 46431 } } peers { permanent_uuid: "8425ada9c46042d9bcdf232b878862ec" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 34443 } } peers { permanent_uuid: "2796bef7589b4d27b8de93c3f83683e4" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 44943 } }
I20260925 07:27:54.132301 19353 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8425ada9c46042d9bcdf232b878862ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8425ada9c46042d9bcdf232b878862ec. Latest consensus state: current_term: 1 leader_uuid: "8425ada9c46042d9bcdf232b878862ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0db28f4d6d194fea809f93c95a24e0af" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 46431 } } peers { permanent_uuid: "8425ada9c46042d9bcdf232b878862ec" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 34443 } } peers { permanent_uuid: "2796bef7589b4d27b8de93c3f83683e4" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 44943 } } }
I20260925 07:27:54.132391 19353 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8425ada9c46042d9bcdf232b878862ec [sys.catalog]: This master's current role is: LEADER
I20260925 07:27:54.132695 19355 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 07:27:54.132907 19355 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 07:27:54.134133 19272 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2796bef7589b4d27b8de93c3f83683e4 [term 1 FOLLOWER]: Refusing update from remote peer 8425ada9c46042d9bcdf232b878862ec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:27:54.134160 19122 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0db28f4d6d194fea809f93c95a24e0af [term 1 FOLLOWER]: Refusing update from remote peer 8425ada9c46042d9bcdf232b878862ec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:27:54.134403 19353 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8425ada9c46042d9bcdf232b878862ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "0db28f4d6d194fea809f93c95a24e0af" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 46431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:27:54.134482 19305 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8425ada9c46042d9bcdf232b878862ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "2796bef7589b4d27b8de93c3f83683e4" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 44943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:27:54.135541 19340 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0db28f4d6d194fea809f93c95a24e0af [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8425ada9c46042d9bcdf232b878862ec. Latest consensus state: current_term: 1 leader_uuid: "8425ada9c46042d9bcdf232b878862ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0db28f4d6d194fea809f93c95a24e0af" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 46431 } } peers { permanent_uuid: "8425ada9c46042d9bcdf232b878862ec" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 34443 } } peers { permanent_uuid: "2796bef7589b4d27b8de93c3f83683e4" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 44943 } } }
I20260925 07:27:54.135552 19326 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2796bef7589b4d27b8de93c3f83683e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8425ada9c46042d9bcdf232b878862ec. Latest consensus state: current_term: 1 leader_uuid: "8425ada9c46042d9bcdf232b878862ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0db28f4d6d194fea809f93c95a24e0af" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 46431 } } peers { permanent_uuid: "8425ada9c46042d9bcdf232b878862ec" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 34443 } } peers { permanent_uuid: "2796bef7589b4d27b8de93c3f83683e4" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 44943 } } }
I20260925 07:27:54.135627 19340 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0db28f4d6d194fea809f93c95a24e0af [sys.catalog]: This master's current role is: FOLLOWER
I20260925 07:27:54.135649 19326 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2796bef7589b4d27b8de93c3f83683e4 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 07:27:54.136693 19353 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8425ada9c46042d9bcdf232b878862ec [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8425ada9c46042d9bcdf232b878862ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0db28f4d6d194fea809f93c95a24e0af" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 46431 } } peers { permanent_uuid: "8425ada9c46042d9bcdf232b878862ec" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 34443 } } peers { permanent_uuid: "2796bef7589b4d27b8de93c3f83683e4" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 44943 } } }
I20260925 07:27:54.136777 19353 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8425ada9c46042d9bcdf232b878862ec [sys.catalog]: This master's current role is: LEADER
I20260925 07:27:54.137212 19326 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2796bef7589b4d27b8de93c3f83683e4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8425ada9c46042d9bcdf232b878862ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0db28f4d6d194fea809f93c95a24e0af" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 46431 } } peers { permanent_uuid: "8425ada9c46042d9bcdf232b878862ec" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 34443 } } peers { permanent_uuid: "2796bef7589b4d27b8de93c3f83683e4" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 44943 } } }
I20260925 07:27:54.137286 19326 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2796bef7589b4d27b8de93c3f83683e4 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 07:27:54.137432 19357 mvcc.cc:205] Tried to move back new op lower bound from 7333155938851438592 to 7333155938845016064. Current Snapshot: MvccSnapshot[applied={T|T < 7333155938851438592}]
I20260925 07:27:54.137555 19356 mvcc.cc:205] Tried to move back new op lower bound from 7333155938851438592 to 7333155938845016064. Current Snapshot: MvccSnapshot[applied={T|T < 7333155938851438592}]
I20260925 07:27:54.137707 19305 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8425ada9c46042d9bcdf232b878862ec [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8425ada9c46042d9bcdf232b878862ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0db28f4d6d194fea809f93c95a24e0af" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 46431 } } peers { permanent_uuid: "8425ada9c46042d9bcdf232b878862ec" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 34443 } } peers { permanent_uuid: "2796bef7589b4d27b8de93c3f83683e4" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 44943 } } }
I20260925 07:27:54.137773 19305 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8425ada9c46042d9bcdf232b878862ec [sys.catalog]: This master's current role is: LEADER
I20260925 07:27:54.138054 19355 catalog_manager.cc:1347] Generated new cluster ID: 3503833ba10341b080368978852cc360
I20260925 07:27:54.138093 19355 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 07:27:54.138108 19340 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0db28f4d6d194fea809f93c95a24e0af [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8425ada9c46042d9bcdf232b878862ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0db28f4d6d194fea809f93c95a24e0af" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 46431 } } peers { permanent_uuid: "8425ada9c46042d9bcdf232b878862ec" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 34443 } } peers { permanent_uuid: "2796bef7589b4d27b8de93c3f83683e4" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 44943 } } }
I20260925 07:27:54.138177 19340 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0db28f4d6d194fea809f93c95a24e0af [sys.catalog]: This master's current role is: FOLLOWER
I20260925 07:27:54.138381 19358 mvcc.cc:205] Tried to move back new op lower bound from 7333155938851438592 to 7333155938845016064. Current Snapshot: MvccSnapshot[applied={T|T < 7333155938851438592}]
I20260925 07:27:54.151199 19355 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 07:27:54.151793 19355 catalog_manager.cc:1504] Loading token signing keys...
I20260925 07:27:54.155557 19355 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8425ada9c46042d9bcdf232b878862ec: Generated new TSK 0
I20260925 07:27:54.155715 19355 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 07:27:54.993587 19337 catalog_manager.cc:1259] Loaded cluster ID: 3503833ba10341b080368978852cc360
I20260925 07:27:54.993639 19337 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2796bef7589b4d27b8de93c3f83683e4: loading cluster ID for follower catalog manager: success
I20260925 07:27:54.994467 19337 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2796bef7589b4d27b8de93c3f83683e4: acquiring CA information for follower catalog manager: success
I20260925 07:27:54.994779 19337 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2796bef7589b4d27b8de93c3f83683e4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 07:27:55.023943 19351 catalog_manager.cc:1259] Loaded cluster ID: 3503833ba10341b080368978852cc360
I20260925 07:27:55.023994 19351 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0db28f4d6d194fea809f93c95a24e0af: loading cluster ID for follower catalog manager: success
I20260925 07:27:55.024796 19351 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0db28f4d6d194fea809f93c95a24e0af: acquiring CA information for follower catalog manager: success
I20260925 07:27:55.025132 19351 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0db28f4d6d194fea809f93c95a24e0af: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 07:27:55.214701 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 19080
I20260925 07:27:55.220042 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 19155
I20260925 07:27:55.224642 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 19230
2026-09-25T07:27:55Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2734 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790321110323796-5753-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 07:27:55 dist-test-slave-1drq krb5kdc[19369](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 07:27:55 dist-test-slave-1drq krb5kdc[19369](info): set up 2 sockets
Sep 25 07:27:55 dist-test-slave-1drq krb5kdc[19369](info): commencing operation
krb5kdc: starting...
W20260925 07:27:56.181030 5753 mini_kdc.cc:121] Time spent starting KDC: real 0.937s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Sep 25 07:27:56 dist-test-slave-1drq krb5kdc[19369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321276, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T07:27:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T07:27:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
I20260925 07:27:56.281985 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:34553
--webserver_interface=127.5.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:35537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.158.126:34553
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:27:56.355280 19382 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:27:56.355444 19382 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:27:56.355465 19382 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:27:56.356575 19382 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:27:56.356618 19382 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:27:56.356634 19382 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:27:56.356663 19382 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:27:56.356693 19382 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:27:56.358040 19382 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:35537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790321110323796-5753-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.158.126:34553
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:34553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19382
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:27:56.358206 19382 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:27:56.358433 19382 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 07:27:56.362938 19382 server_base.cc:1034] running on GCE node
W20260925 07:27:56.362919 19388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:27:56.362907 19387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:27:56.362907 19390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:27:56.363358 19382 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:27:56.363667 19382 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:27:56.364818 19382 hybrid_clock.cc:648] HybridClock initialized: now 1790321276364838 us; error 60 us; skew 500 ppm
Sep 25 07:27:56 dist-test-slave-1drq krb5kdc[19369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321276, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:27:56.367141 19382 init.cc:377] Logged in from keytab as kudu/127.5.158.126@KRBTEST.COM (short username kudu)
I20260925 07:27:56.367537 19382 webserver.cc:460] Webserver started at http://127.5.158.126:42889/ using document root <none> and password file <none>
I20260925 07:27:56.367729 19382 fs_manager.cc:359] Metadata directory not provided
I20260925 07:27:56.367789 19382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:27:56.367906 19382 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:27:56.368779 19382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790321110323796-5753-0/minicluster-data/master-0/data/instance:
uuid: "2dcb2a9adf814f82bd735140e42cabf6"
format_stamp: "Formatted at 2026-09-25 07:27:56 on dist-test-slave-1drq"
server_key: "2912bdbbb81e252c8fef027bf7a81aae"
server_key_iv: "84406242a66eafe0e509ee977d279ee1"
server_key_version: "encryptionkey@0"
I20260925 07:27:56.369146 19382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790321110323796-5753-0/minicluster-data/master-0/wal/instance:
uuid: "2dcb2a9adf814f82bd735140e42cabf6"
format_stamp: "Formatted at 2026-09-25 07:27:56 on dist-test-slave-1drq"
server_key: "2912bdbbb81e252c8fef027bf7a81aae"
server_key_iv: "84406242a66eafe0e509ee977d279ee1"
server_key_version: "encryptionkey@0"
I20260925 07:27:56.370317 19382 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 07:27:56.370963 19397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:56.371138 19382 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 07:27:56.371225 19382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "2dcb2a9adf814f82bd735140e42cabf6"
format_stamp: "Formatted at 2026-09-25 07:27:56 on dist-test-slave-1drq"
server_key: "2912bdbbb81e252c8fef027bf7a81aae"
server_key_iv: "84406242a66eafe0e509ee977d279ee1"
server_key_version: "encryptionkey@0"
I20260925 07:27:56.371299 19382 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:27:56.386608 19382 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:27:56.388079 19382 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:27:56.388217 19382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:27:56.391772 19382 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.126:34553
I20260925 07:27:56.391810 19449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.126:34553 every 8 connection(s)
I20260925 07:27:56.392105 19382 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:27:56.392853 19450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:27:56.394930 19450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2dcb2a9adf814f82bd735140e42cabf6: Bootstrap starting.
I20260925 07:27:56.395478 19450 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2dcb2a9adf814f82bd735140e42cabf6: Neither blocks nor log segments found. Creating new log.
I20260925 07:27:56.395537 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 19382
I20260925 07:27:56.395632 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790321110323796-5753-0/minicluster-data/master-0/wal/instance
I20260925 07:27:56.395843 5753 external_mini_cluster.cc:1428] Setting key 0338979192340f06a5c52851dd823084
I20260925 07:27:56.395829 19450 log.cc:824] T 00000000000000000000000000000000 P 2dcb2a9adf814f82bd735140e42cabf6: Log is configured to *not* fsync() on all Append() calls
I20260925 07:27:56.396620 19450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2dcb2a9adf814f82bd735140e42cabf6: No bootstrap required, opened a new log
I20260925 07:27:56.398730 19450 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2dcb2a9adf814f82bd735140e42cabf6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dcb2a9adf814f82bd735140e42cabf6" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 34553 } }
I20260925 07:27:56.398873 19450 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2dcb2a9adf814f82bd735140e42cabf6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:27:56.398905 19450 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2dcb2a9adf814f82bd735140e42cabf6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2dcb2a9adf814f82bd735140e42cabf6, State: Initialized, Role: FOLLOWER
I20260925 07:27:56.399015 19450 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2dcb2a9adf814f82bd735140e42cabf6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dcb2a9adf814f82bd735140e42cabf6" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 34553 } }
I20260925 07:27:56.399085 19450 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2dcb2a9adf814f82bd735140e42cabf6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 07:27:56.399111 19450 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2dcb2a9adf814f82bd735140e42cabf6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 07:27:56.399150 19450 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2dcb2a9adf814f82bd735140e42cabf6 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:27:56.399709 19450 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2dcb2a9adf814f82bd735140e42cabf6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dcb2a9adf814f82bd735140e42cabf6" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 34553 } }
I20260925 07:27:56.399832 19450 leader_election.cc:302] T 00000000000000000000000000000000 P 2dcb2a9adf814f82bd735140e42cabf6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2dcb2a9adf814f82bd735140e42cabf6; no voters:
I20260925 07:27:56.399988 19450 leader_election.cc:288] T 00000000000000000000000000000000 P 2dcb2a9adf814f82bd735140e42cabf6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 07:27:56.400040 19455 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2dcb2a9adf814f82bd735140e42cabf6 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:27:56.400215 19450 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2dcb2a9adf814f82bd735140e42cabf6 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:27:56.400257 19455 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2dcb2a9adf814f82bd735140e42cabf6 [term 1 LEADER]: Becoming Leader. State: Replica: 2dcb2a9adf814f82bd735140e42cabf6, State: Running, Role: LEADER
I20260925 07:27:56.400436 19455 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2dcb2a9adf814f82bd735140e42cabf6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dcb2a9adf814f82bd735140e42cabf6" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 34553 } }
I20260925 07:27:56.401413 19456 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dcb2a9adf814f82bd735140e42cabf6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2dcb2a9adf814f82bd735140e42cabf6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dcb2a9adf814f82bd735140e42cabf6" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 34553 } } }
I20260925 07:27:56.401520 19456 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dcb2a9adf814f82bd735140e42cabf6 [sys.catalog]: This master's current role is: LEADER
I20260925 07:27:56.401578 19457 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dcb2a9adf814f82bd735140e42cabf6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2dcb2a9adf814f82bd735140e42cabf6. Latest consensus state: current_term: 1 leader_uuid: "2dcb2a9adf814f82bd735140e42cabf6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dcb2a9adf814f82bd735140e42cabf6" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 34553 } } }
I20260925 07:27:56.401640 19457 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dcb2a9adf814f82bd735140e42cabf6 [sys.catalog]: This master's current role is: LEADER
W20260925 07:27:56.402315 19469 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2dcb2a9adf814f82bd735140e42cabf6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 07:27:56.402367 19469 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 07:27:56.402413 19470 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 07:27:56.402576 19470 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 25 07:27:56 dist-test-slave-1drq krb5kdc[19369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321276, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:27:56.403947 19470 catalog_manager.cc:1347] Generated new cluster ID: 63133a0e465b4e039fdeff7ce2f177a1
I20260925 07:27:56.404006 19470 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 07:27:56.406229 19454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:56.396331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44978 (local address 127.5.158.126:34553)
0925 07:27:56.396466 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:56.396478 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:56.396523 (+ 45us) server_negotiation.cc:407] Connection header received
0925 07:27:56.396925 (+ 402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:56.396931 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:56.396973 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:56.397073 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:27:56.397794 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:56.398586 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:56.399563 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:56.400032 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:56.400152 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:56.403127 (+ 2975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:27:56.403131 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:27:56.403131 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:27:56.403141 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:27:56.404825 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:56.405223 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:56.405226 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:56.405226 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:56.405309 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:56.405520 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:56.405521 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:56.405521 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:56.405712 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:27:56.405848 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:56.405939 (+ 91us) server_negotiation.cc:299] Negotiation successful
0925 07:27:56.406074 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":65,"threads_started":1}
I20260925 07:27:56.409823 5753 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 07:27:56.410661 19470 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 07:27:56.411248 19470 catalog_manager.cc:1504] Loading token signing keys...
I20260925 07:27:56.416796 19470 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2dcb2a9adf814f82bd735140e42cabf6: Generated new TSK 0
I20260925 07:27:56.416934 19470 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 07:27:56.463445 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 19382
2026-09-25T07:27:56Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1236 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 07:27:56 dist-test-slave-1drq krb5kdc[19477](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 07:27:56 dist-test-slave-1drq krb5kdc[19477](info): set up 2 sockets
Sep 25 07:27:56 dist-test-slave-1drq krb5kdc[19477](info): commencing operation
krb5kdc: starting...
W20260925 07:27:57.428581 5753 mini_kdc.cc:121] Time spent starting KDC: real 0.948s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Sep 25 07:27:57 dist-test-slave-1drq krb5kdc[19477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T07:27:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T07:27:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
I20260925 07:27:57.529675 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:43241
--webserver_interface=127.5.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:42351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.158.126:43241
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:27:57.601848 19490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:27:57.602025 19490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:27:57.602047 19490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:27:57.603106 19490 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:27:57.603142 19490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:27:57.603155 19490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:27:57.603169 19490 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:27:57.603180 19490 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:27:57.604517 19490 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:42351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.158.126:43241
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:43241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19490
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:27:57.604692 19490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:27:57.604842 19490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:27:57.608906 19495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:27:57.609045 19490 server_base.cc:1034] running on GCE node
W20260925 07:27:57.608891 19498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:27:57.608886 19496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:27:57.609462 19490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:27:57.609673 19490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:27:57.610800 19490 hybrid_clock.cc:648] HybridClock initialized: now 1790321277610788 us; error 26 us; skew 500 ppm
Sep 25 07:27:57 dist-test-slave-1drq krb5kdc[19477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321277, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:27:57.612874 19490 init.cc:377] Logged in from keytab as kudu/127.5.158.126@KRBTEST.COM (short username kudu)
I20260925 07:27:57.613294 19490 webserver.cc:460] Webserver started at http://127.5.158.126:43063/ using document root <none> and password file <none>
I20260925 07:27:57.613500 19490 fs_manager.cc:359] Metadata directory not provided
I20260925 07:27:57.613562 19490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:27:57.613699 19490 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:27:57.614674 19490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/master-0/data/instance:
uuid: "e8e5d0766eb541f5b6c08f1844b6d6e9"
format_stamp: "Formatted at 2026-09-25 07:27:57 on dist-test-slave-1drq"
server_key: "489dfa6637e0085f04459f57da95f3a2"
server_key_iv: "05622286d32ccad4a4b13f49ad505c53"
server_key_version: "encryptionkey@0"
I20260925 07:27:57.615051 19490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/master-0/wal/instance:
uuid: "e8e5d0766eb541f5b6c08f1844b6d6e9"
format_stamp: "Formatted at 2026-09-25 07:27:57 on dist-test-slave-1drq"
server_key: "489dfa6637e0085f04459f57da95f3a2"
server_key_iv: "05622286d32ccad4a4b13f49ad505c53"
server_key_version: "encryptionkey@0"
I20260925 07:27:57.616209 19490 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 07:27:57.616851 19505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:57.617007 19490 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:27:57.617118 19490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "e8e5d0766eb541f5b6c08f1844b6d6e9"
format_stamp: "Formatted at 2026-09-25 07:27:57 on dist-test-slave-1drq"
server_key: "489dfa6637e0085f04459f57da95f3a2"
server_key_iv: "05622286d32ccad4a4b13f49ad505c53"
server_key_version: "encryptionkey@0"
I20260925 07:27:57.617223 19490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:27:57.634402 19490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:27:57.636528 19490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:27:57.636680 19490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:27:57.640376 19490 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.126:43241
I20260925 07:27:57.640419 19557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.126:43241 every 8 connection(s)
I20260925 07:27:57.640714 19490 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:27:57.641748 19558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:27:57.643386 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 19490
I20260925 07:27:57.643498 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/master-0/wal/instance
I20260925 07:27:57.643682 5753 external_mini_cluster.cc:1428] Setting key 62b7d04c1dca22752e6fb57df0bfd988
I20260925 07:27:57.643986 19558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9: Bootstrap starting.
I20260925 07:27:57.644876 19558 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9: Neither blocks nor log segments found. Creating new log.
I20260925 07:27:57.645210 19558 log.cc:824] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9: Log is configured to *not* fsync() on all Append() calls
I20260925 07:27:57.645859 19558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9: No bootstrap required, opened a new log
I20260925 07:27:57.647723 19558 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8e5d0766eb541f5b6c08f1844b6d6e9" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 43241 } }
I20260925 07:27:57.647861 19558 raft_consensus.cc:374] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:27:57.647888 19558 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8e5d0766eb541f5b6c08f1844b6d6e9, State: Initialized, Role: FOLLOWER
I20260925 07:27:57.647986 19558 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8e5d0766eb541f5b6c08f1844b6d6e9" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 43241 } }
I20260925 07:27:57.648056 19558 raft_consensus.cc:388] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 07:27:57.648084 19558 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 07:27:57.648128 19558 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:27:57.648720 19558 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8e5d0766eb541f5b6c08f1844b6d6e9" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 43241 } }
I20260925 07:27:57.648798 19558 leader_election.cc:302] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e8e5d0766eb541f5b6c08f1844b6d6e9; no voters:
I20260925 07:27:57.648952 19558 leader_election.cc:288] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 07:27:57.648970 19563 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:27:57.649070 19563 raft_consensus.cc:686] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9 [term 1 LEADER]: Becoming Leader. State: Replica: e8e5d0766eb541f5b6c08f1844b6d6e9, State: Running, Role: LEADER
I20260925 07:27:57.649200 19558 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:27:57.649230 19563 consensus_queue.cc:237] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8e5d0766eb541f5b6c08f1844b6d6e9" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 43241 } }
I20260925 07:27:57.649597 19563 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8e5d0766eb541f5b6c08f1844b6d6e9. Latest consensus state: current_term: 1 leader_uuid: "e8e5d0766eb541f5b6c08f1844b6d6e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8e5d0766eb541f5b6c08f1844b6d6e9" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 43241 } } }
I20260925 07:27:57.649731 19563 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9 [sys.catalog]: This master's current role is: LEADER
Sep 25 07:27:57 dist-test-slave-1drq krb5kdc[19477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:27:57.650122 19572 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 07:27:57.650311 19565 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e8e5d0766eb541f5b6c08f1844b6d6e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8e5d0766eb541f5b6c08f1844b6d6e9" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 43241 } } }
I20260925 07:27:57.650373 19565 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9 [sys.catalog]: This master's current role is: LEADER
I20260925 07:27:57.650552 19572 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 07:27:57.651952 19572 catalog_manager.cc:1347] Generated new cluster ID: 6be6538bdebe4e52a9afcf87a34de8dd
I20260925 07:27:57.651996 19572 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 07:27:57.653862 19561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:57.644192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36136 (local address 127.5.158.126:43241)
0925 07:27:57.644343 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:57.644352 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:57.644382 (+ 30us) server_negotiation.cc:407] Connection header received
0925 07:27:57.645008 (+ 626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:57.645017 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:57.645081 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:57.645255 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:27:57.645806 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:57.646959 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:57.647568 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:57.648079 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:57.648185 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:57.650845 (+ 2660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:27:57.650849 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:27:57.650849 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:27:57.650866 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:27:57.652468 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:57.652787 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:57.652789 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:57.652789 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:57.652870 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:57.653056 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:57.653057 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:57.653058 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:57.653279 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:27:57.653411 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:57.653507 (+ 96us) server_negotiation.cc:299] Negotiation successful
0925 07:27:57.653624 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":80,"threads_started":1}
I20260925 07:27:57.673480 19572 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 07:27:57.673959 19572 catalog_manager.cc:1504] Loading token signing keys...
I20260925 07:27:57.684473 19572 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9: Generated new TSK 0
I20260925 07:27:57.684644 19572 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.158.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab.
Entry for principal kudu/127.5.158.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab.
I20260925 07:27:57.715950 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.65:0
--local_ip_for_outbound_sockets=127.5.158.65
--webserver_interface=127.5.158.65
--webserver_port=0
--tserver_master_addrs=127.5.158.126:43241
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab
--principal=kudu/127.5.158.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:42351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:27:57.800134 19584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:27:57.800307 19584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:27:57.800326 19584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:27:57.801440 19584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:27:57.801478 19584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:27:57.801504 19584 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.65
I20260925 07:27:57.802834 19584 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:42351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab
--principal=kudu/127.5.158.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.158.65
--webserver_port=0
--tserver_master_addrs=127.5.158.126:43241
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19584
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.65
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:27:57.803002 19584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:27:57.803158 19584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:27:57.807776 19589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:27:57.807857 19590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:27:57.807874 19584 server_base.cc:1034] running on GCE node
W20260925 07:27:57.807753 19592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:27:57.808089 19584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:27:57.808287 19584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:27:57.809413 19584 hybrid_clock.cc:648] HybridClock initialized: now 1790321277809403 us; error 26 us; skew 500 ppm
Sep 25 07:27:57 dist-test-slave-1drq krb5kdc[19477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321277, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:27:57.811728 19584 init.cc:377] Logged in from keytab as kudu/127.5.158.65@KRBTEST.COM (short username kudu)
I20260925 07:27:57.812104 19584 webserver.cc:460] Webserver started at http://127.5.158.65:35217/ using document root <none> and password file <none>
I20260925 07:27:57.812319 19584 fs_manager.cc:359] Metadata directory not provided
I20260925 07:27:57.812361 19584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:27:57.812485 19584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:27:57.813441 19584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-0/data/instance:
uuid: "de941b832d3d46fc82275294327a08ac"
format_stamp: "Formatted at 2026-09-25 07:27:57 on dist-test-slave-1drq"
server_key: "cf9e5c9c55e1d3ec00bba553e70b789d"
server_key_iv: "0eb4622a47088c9d8b19c21bf82e4729"
server_key_version: "encryptionkey@0"
I20260925 07:27:57.813813 19584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-0/wal/instance:
uuid: "de941b832d3d46fc82275294327a08ac"
format_stamp: "Formatted at 2026-09-25 07:27:57 on dist-test-slave-1drq"
server_key: "cf9e5c9c55e1d3ec00bba553e70b789d"
server_key_iv: "0eb4622a47088c9d8b19c21bf82e4729"
server_key_version: "encryptionkey@0"
I20260925 07:27:57.815130 19584 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 07:27:57.815826 19599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:57.815995 19584 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:27:57.816094 19584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-0/wal
uuid: "de941b832d3d46fc82275294327a08ac"
format_stamp: "Formatted at 2026-09-25 07:27:57 on dist-test-slave-1drq"
server_key: "cf9e5c9c55e1d3ec00bba553e70b789d"
server_key_iv: "0eb4622a47088c9d8b19c21bf82e4729"
server_key_version: "encryptionkey@0"
I20260925 07:27:57.816172 19584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:27:57.835017 19584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:27:57.836690 19584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:27:57.836839 19584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:27:57.837054 19584 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:27:57.837343 19584 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:27:57.837396 19584 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:57.837433 19584 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:27:57.837468 19584 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:57.842971 19584 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.65:35035
I20260925 07:27:57.843007 19712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.65:35035 every 8 connection(s)
I20260925 07:27:57.843326 19584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
Sep 25 07:27:57 dist-test-slave-1drq krb5kdc[19477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321277, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.65@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:27:57.850301 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 19584
I20260925 07:27:57.850423 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-0/wal/instance
I20260925 07:27:57.850610 5753 external_mini_cluster.cc:1428] Setting key e5b476b67fcbf9c62a918f79cd2152b7
I20260925 07:27:57.852102 19561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:57.844067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:42747 (local address 127.5.158.126:43241)
0925 07:27:57.844135 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:57.844143 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:57.844360 (+ 217us) server_negotiation.cc:407] Connection header received
0925 07:27:57.844870 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:57.844872 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:57.844887 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:57.844956 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:27:57.845902 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:57.846364 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:57.847284 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:57.847478 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:57.847569 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:57.849457 (+ 1888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:27:57.849459 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:27:57.849459 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:27:57.849466 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:27:57.850768 (+ 1302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:57.851213 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:57.851215 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:57.851215 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:57.851278 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:57.851552 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:57.851553 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:57.851553 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:57.851641 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:27:57.851696 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:57.851988 (+ 292us) server_negotiation.cc:299] Negotiation successful
0925 07:27:57.852047 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260925 07:27:57.852350 19715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:57.844043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.65:42747)
0925 07:27:57.844241 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:57.844275 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 07:27:57.844756 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:27:57.845014 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:27:57.845017 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:27:57.845606 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:27:57.845789 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:57.845793 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:57.846495 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:57.846496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:57.847185 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:57.847187 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:57.847578 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:57.847579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:57.847696 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:57.848073 (+ 377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:27:57.848080 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:27:57.849352 (+ 1272us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:27:57.850860 (+ 1508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:27:57.850866 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:27:57.850868 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:27:57.851101 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:27:57.851350 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:27:57.851351 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:27:57.851351 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:27:57.851482 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:27:57.851711 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:27:57.851718 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:27:57.851863 (+ 145us) client_negotiation.cc:769] Sending connection context
0925 07:27:57.851972 (+ 109us) client_negotiation.cc:240] Negotiation successful
0925 07:27:57.852122 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":74,"threads_started":1}
I20260925 07:27:57.852814 19713 heartbeater.cc:347] Connected to a master server at 127.5.158.126:43241
I20260925 07:27:57.852891 19713 heartbeater.cc:464] Registering TS with master...
I20260925 07:27:57.853041 19713 heartbeater.cc:511] Master 127.5.158.126:43241 requested a full tablet report, sending...
I20260925 07:27:57.853487 19522 ts_manager.cc:194] Registered new tserver with Master: de941b832d3d46fc82275294327a08ac (127.5.158.65:35035)
I20260925 07:27:57.854061 19522 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.65@KRBTEST.COM'} at 127.5.158.65:42747
WARNING: no policy specified for kudu/127.5.158.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab.
Entry for principal kudu/127.5.158.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab.
I20260925 07:27:57.875222 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.66:0
--local_ip_for_outbound_sockets=127.5.158.66
--webserver_interface=127.5.158.66
--webserver_port=0
--tserver_master_addrs=127.5.158.126:43241
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab
--principal=kudu/127.5.158.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:42351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:27:57.949329 19718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:27:57.949510 19718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:27:57.949543 19718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:27:57.950601 19718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:27:57.950661 19718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:27:57.950703 19718 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.66
I20260925 07:27:57.952009 19718 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:42351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab
--principal=kudu/127.5.158.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.158.66
--webserver_port=0
--tserver_master_addrs=127.5.158.126:43241
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19718
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.66
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:27:57.952255 19718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:27:57.952451 19718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:27:57.956883 19723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:27:57.956959 19724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:27:57.957129 19726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:27:57.957075 19718 server_base.cc:1034] running on GCE node
I20260925 07:27:57.957301 19718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:27:57.957532 19718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:27:57.958688 19718 hybrid_clock.cc:648] HybridClock initialized: now 1790321277958675 us; error 31 us; skew 500 ppm
Sep 25 07:27:57 dist-test-slave-1drq krb5kdc[19477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321277, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:27:57.961061 19718 init.cc:377] Logged in from keytab as kudu/127.5.158.66@KRBTEST.COM (short username kudu)
I20260925 07:27:57.961534 19718 webserver.cc:460] Webserver started at http://127.5.158.66:42683/ using document root <none> and password file <none>
I20260925 07:27:57.961705 19718 fs_manager.cc:359] Metadata directory not provided
I20260925 07:27:57.961742 19718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:27:57.961822 19718 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:27:57.962649 19718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-1/data/instance:
uuid: "7cce4a7fbbd34fbd86be2ab1c4671796"
format_stamp: "Formatted at 2026-09-25 07:27:57 on dist-test-slave-1drq"
server_key: "b1699fda57649e2a8ab6e67583325ffc"
server_key_iv: "335357c6c2f5b66a3fb7cc7d82052fcf"
server_key_version: "encryptionkey@0"
I20260925 07:27:57.962936 19718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-1/wal/instance:
uuid: "7cce4a7fbbd34fbd86be2ab1c4671796"
format_stamp: "Formatted at 2026-09-25 07:27:57 on dist-test-slave-1drq"
server_key: "b1699fda57649e2a8ab6e67583325ffc"
server_key_iv: "335357c6c2f5b66a3fb7cc7d82052fcf"
server_key_version: "encryptionkey@0"
I20260925 07:27:57.964066 19718 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260925 07:27:57.964695 19733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:57.964870 19718 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:57.964988 19718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-1/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-1/wal
uuid: "7cce4a7fbbd34fbd86be2ab1c4671796"
format_stamp: "Formatted at 2026-09-25 07:27:57 on dist-test-slave-1drq"
server_key: "b1699fda57649e2a8ab6e67583325ffc"
server_key_iv: "335357c6c2f5b66a3fb7cc7d82052fcf"
server_key_version: "encryptionkey@0"
I20260925 07:27:57.965065 19718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:27:57.972366 19718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:27:57.973927 19718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:27:57.974076 19718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:27:57.974268 19718 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:27:57.974550 19718 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:27:57.974606 19718 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:57.974642 19718 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:27:57.974670 19718 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:57.980252 19718 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.66:38307
I20260925 07:27:57.980295 19846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.66:38307 every 8 connection(s)
I20260925 07:27:57.980542 19718 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
Sep 25 07:27:57 dist-test-slave-1drq krb5kdc[19477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321277, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.66@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:27:57.988278 19561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:57.981201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:41577 (local address 127.5.158.126:43241)
0925 07:27:57.981250 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:57.981253 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:57.981479 (+ 226us) server_negotiation.cc:407] Connection header received
0925 07:27:57.981887 (+ 408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:57.981888 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:57.981898 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:57.981948 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:27:57.982686 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:57.983124 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:57.983984 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:57.984177 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:57.984227 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:57.986016 (+ 1789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:27:57.986017 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:27:57.986017 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:27:57.986023 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:27:57.987302 (+ 1279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:57.987595 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:57.987596 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:57.987596 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:57.987641 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:57.987829 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:57.987830 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:57.987830 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:57.987916 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:27:57.987960 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:57.988168 (+ 208us) server_negotiation.cc:299] Negotiation successful
0925 07:27:57.988221 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 07:27:57.988341 19849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:57.981183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.66:41577)
0925 07:27:57.981337 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:57.981368 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0925 07:27:57.981798 (+ 430us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:27:57.981991 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:27:57.981994 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:27:57.982470 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:27:57.982622 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:57.982626 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:57.983203 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:57.983203 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:57.983912 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:57.983914 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:57.984246 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:57.984246 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:57.984334 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:57.984703 (+ 369us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:27:57.984710 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:27:57.985936 (+ 1226us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:27:57.987364 (+ 1428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:27:57.987368 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:27:57.987369 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:27:57.987539 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:27:57.987697 (+ 158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:27:57.987697 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:27:57.987697 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:27:57.987774 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:27:57.987966 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:27:57.987969 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:27:57.988057 (+ 88us) client_negotiation.cc:769] Sending connection context
0925 07:27:57.988126 (+ 69us) client_negotiation.cc:240] Negotiation successful
0925 07:27:57.988220 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":81,"threads_started":1}
I20260925 07:27:57.988937 19847 heartbeater.cc:347] Connected to a master server at 127.5.158.126:43241
I20260925 07:27:57.988950 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 19718
I20260925 07:27:57.989051 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-1/wal/instance
I20260925 07:27:57.989061 19847 heartbeater.cc:464] Registering TS with master...
I20260925 07:27:57.989284 19847 heartbeater.cc:511] Master 127.5.158.126:43241 requested a full tablet report, sending...
I20260925 07:27:57.989419 5753 external_mini_cluster.cc:1428] Setting key 9b43b5f07d4eb400a09ccc5fa91875d6
I20260925 07:27:57.989755 19522 ts_manager.cc:194] Registered new tserver with Master: 7cce4a7fbbd34fbd86be2ab1c4671796 (127.5.158.66:38307)
I20260925 07:27:57.990254 19522 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.66@KRBTEST.COM'} at 127.5.158.66:41577
WARNING: no policy specified for kudu/127.5.158.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab.
Entry for principal kudu/127.5.158.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab.
I20260925 07:27:58.015519 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.67:0
--local_ip_for_outbound_sockets=127.5.158.67
--webserver_interface=127.5.158.67
--webserver_port=0
--tserver_master_addrs=127.5.158.126:43241
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab
--principal=kudu/127.5.158.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:42351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:27:58.087353 19852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:27:58.087560 19852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:27:58.087594 19852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:27:58.088677 19852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:27:58.088743 19852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:27:58.088788 19852 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.67
I20260925 07:27:58.090188 19852 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:42351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab
--principal=kudu/127.5.158.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.158.67
--webserver_port=0
--tserver_master_addrs=127.5.158.126:43241
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19852
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.67
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:27:58.090421 19852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:27:58.090618 19852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:27:58.094978 19857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:27:58.095021 19858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:27:58.095118 19852 server_base.cc:1034] running on GCE node
W20260925 07:27:58.095206 19860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:27:58.095387 19852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:27:58.095556 19852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:27:58.096685 19852 hybrid_clock.cc:648] HybridClock initialized: now 1790321278096670 us; error 31 us; skew 500 ppm
Sep 25 07:27:58 dist-test-slave-1drq krb5kdc[19477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321278, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:27:58.098907 19852 init.cc:377] Logged in from keytab as kudu/127.5.158.67@KRBTEST.COM (short username kudu)
I20260925 07:27:58.099288 19852 webserver.cc:460] Webserver started at http://127.5.158.67:43081/ using document root <none> and password file <none>
I20260925 07:27:58.099503 19852 fs_manager.cc:359] Metadata directory not provided
I20260925 07:27:58.099567 19852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:27:58.099712 19852 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:27:58.100597 19852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-2/data/instance:
uuid: "308d38b0e2fd42d7afe17e8340b87f16"
format_stamp: "Formatted at 2026-09-25 07:27:58 on dist-test-slave-1drq"
server_key: "21cc47f0abc344fa8483396a74e5c19b"
server_key_iv: "71f98021aca893db8c042fb45cc56cda"
server_key_version: "encryptionkey@0"
I20260925 07:27:58.100965 19852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-2/wal/instance:
uuid: "308d38b0e2fd42d7afe17e8340b87f16"
format_stamp: "Formatted at 2026-09-25 07:27:58 on dist-test-slave-1drq"
server_key: "21cc47f0abc344fa8483396a74e5c19b"
server_key_iv: "71f98021aca893db8c042fb45cc56cda"
server_key_version: "encryptionkey@0"
I20260925 07:27:58.102241 19852 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 07:27:58.102906 19867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:58.103070 19852 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 07:27:58.103168 19852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-2/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-2/wal
uuid: "308d38b0e2fd42d7afe17e8340b87f16"
format_stamp: "Formatted at 2026-09-25 07:27:58 on dist-test-slave-1drq"
server_key: "21cc47f0abc344fa8483396a74e5c19b"
server_key_iv: "71f98021aca893db8c042fb45cc56cda"
server_key_version: "encryptionkey@0"
I20260925 07:27:58.103262 19852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:27:58.113617 19852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:27:58.115227 19852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:27:58.115372 19852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:27:58.115558 19852 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:27:58.115820 19852 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:27:58.115869 19852 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:58.115899 19852 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:27:58.115923 19852 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:58.121560 19852 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.67:35777
I20260925 07:27:58.121592 19980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.67:35777 every 8 connection(s)
I20260925 07:27:58.121898 19852 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
Sep 25 07:27:58 dist-test-slave-1drq krb5kdc[19477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321278, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.67@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:27:58.129274 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 19852
I20260925 07:27:58.129408 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-2/wal/instance
I20260925 07:27:58.129591 5753 external_mini_cluster.cc:1428] Setting key 0be66dda81e96ed0aea913405ecfebb1
I20260925 07:27:58.130442 19561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:58.122560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:53285 (local address 127.5.158.126:43241)
0925 07:27:58.122642 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:58.122644 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:58.122771 (+ 127us) server_negotiation.cc:407] Connection header received
0925 07:27:58.123238 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:58.123240 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:58.123251 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:58.123301 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:27:58.124133 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:58.124623 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:58.125648 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:58.125838 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:58.125922 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:58.127642 (+ 1720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:27:58.127643 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:27:58.127643 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:27:58.127649 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:27:58.128907 (+ 1258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:58.129320 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:58.129321 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:58.129321 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:58.129387 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:58.129677 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:58.129678 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:58.129678 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:58.129826 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:27:58.129879 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:58.130288 (+ 409us) server_negotiation.cc:299] Negotiation successful
0925 07:27:58.130361 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260925 07:27:58.130616 19983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:58.122494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.67:53285)
0925 07:27:58.122646 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:58.122687 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 07:27:58.123106 (+ 419us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:27:58.123358 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:27:58.123362 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:27:58.123876 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:27:58.124036 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:58.124039 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:58.124725 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:58.124726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:58.125551 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:58.125553 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:58.125932 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:58.125933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:58.126045 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:58.126412 (+ 367us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:27:58.126419 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:27:58.127561 (+ 1142us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:27:58.128978 (+ 1417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:27:58.128983 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:27:58.128985 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:27:58.129248 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:27:58.129480 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:27:58.129481 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:27:58.129482 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:27:58.129586 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:27:58.129968 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:27:58.129973 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:27:58.130133 (+ 160us) client_negotiation.cc:769] Sending connection context
0925 07:27:58.130251 (+ 118us) client_negotiation.cc:240] Negotiation successful
0925 07:27:58.130406 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":63,"threads_started":1}
I20260925 07:27:58.131018 19981 heartbeater.cc:347] Connected to a master server at 127.5.158.126:43241
I20260925 07:27:58.131093 19981 heartbeater.cc:464] Registering TS with master...
I20260925 07:27:58.131212 19981 heartbeater.cc:511] Master 127.5.158.126:43241 requested a full tablet report, sending...
I20260925 07:27:58.131517 19522 ts_manager.cc:194] Registered new tserver with Master: 308d38b0e2fd42d7afe17e8340b87f16 (127.5.158.67:35777)
I20260925 07:27:58.131827 19522 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.67@KRBTEST.COM'} at 127.5.158.67:53285
I20260925 07:27:58.141479 5753 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 07:27:58.148041 19561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:58.142705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36142 (local address 127.5.158.126:43241)
0925 07:27:58.142772 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:58.142774 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:58.142873 (+ 99us) server_negotiation.cc:407] Connection header received
0925 07:27:58.142939 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:58.142940 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:58.142949 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:58.143013 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:27:58.143540 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:58.143999 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:58.144733 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:58.144914 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:58.145008 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:58.146037 (+ 1029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:27:58.146038 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:27:58.146038 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:27:58.146044 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:27:58.147002 (+ 958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:58.147325 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:58.147326 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:58.147326 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:58.147377 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:58.147560 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:58.147561 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:58.147561 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:58.147689 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:27:58.147790 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:58.147889 (+ 99us) server_negotiation.cc:299] Negotiation successful
0925 07:27:58.147954 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260925 07:27:58.149408 19522 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36142:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 07:27:58.151067 19522 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 07:27:58.156639 19990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:58.152994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.66:38307 (local address 127.0.0.1:54578)
0925 07:27:58.153257 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:58.153290 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0925 07:27:58.153344 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:27:58.153889 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:27:58.153890 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:27:58.153892 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:27:58.153938 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:58.153940 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:58.154846 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:58.154847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:58.155561 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:58.155562 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:58.156353 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:58.156354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:58.156427 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:58.156439 (+ 12us) client_negotiation.cc:769] Sending connection context
0925 07:27:58.156530 (+ 91us) client_negotiation.cc:240] Negotiation successful
0925 07:27:58.156578 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":56,"threads_started":1}
I20260925 07:27:58.156670 19991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:58.153099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54578 (local address 127.5.158.66:38307)
0925 07:27:58.153708 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:58.153712 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:58.153721 (+ 9us) server_negotiation.cc:407] Connection header received
0925 07:27:58.153775 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:58.153777 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:58.153792 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:58.154014 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:27:58.154093 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:58.154746 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:58.155741 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:58.156252 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:58.156338 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:58.156393 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:58.156548 (+ 155us) server_negotiation.cc:299] Negotiation successful
0925 07:27:58.156591 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":68,"threads_started":1}
I20260925 07:27:58.157385 19992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:58.152970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.67:35777 (local address 127.0.0.1:46690)
0925 07:27:58.153254 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:58.153287 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0925 07:27:58.153358 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:27:58.153610 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:27:58.153612 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:27:58.153615 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:27:58.153871 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:58.153873 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:58.155137 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:58.155138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:58.156249 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:58.156249 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:58.157143 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:58.157144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:58.157207 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:58.157210 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 07:27:58.157257 (+ 47us) client_negotiation.cc:240] Negotiation successful
0925 07:27:58.157297 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"mutex_wait_us":51,"thread_start_us":42,"threads_started":1}
I20260925 07:27:58.157385 19993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:58.153118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46690 (local address 127.5.158.67:35777)
0925 07:27:58.153430 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:58.153435 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:58.153445 (+ 10us) server_negotiation.cc:407] Connection header received
0925 07:27:58.153475 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:58.153477 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:58.153490 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:58.153600 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:27:58.153956 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:58.155033 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:58.156355 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:58.156978 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:58.157020 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:58.157065 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:58.157283 (+ 218us) server_negotiation.cc:299] Negotiation successful
0925 07:27:58.157321 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":72,"threads_started":1}
I20260925 07:27:58.157476 19989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:58.153152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:35035 (local address 127.0.0.1:60828)
0925 07:27:58.153422 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:58.153431 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 07:27:58.153475 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:27:58.153780 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:27:58.153781 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:27:58.153783 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:27:58.153852 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:58.153853 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:58.155137 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:58.155138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:58.156247 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:58.156249 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:58.157288 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:58.157289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:58.157333 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:58.157336 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 07:27:58.157377 (+ 41us) client_negotiation.cc:240] Negotiation successful
0925 07:27:58.157411 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"mutex_wait_us":49,"thread_start_us":30,"threads_started":1}
I20260925 07:27:58.157411 19781 tablet_service.cc:1467] Processing CreateTablet for tablet f82d9531baa24f5ea83e65a0b12c4c71 (DEFAULT_TABLE table=test-table [id=e5f5618858f14d2f89eda69c6d461c12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:27:58.157683 19994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:58.153255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60828 (local address 127.5.158.65:35035)
0925 07:27:58.153425 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:58.153429 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:58.153470 (+ 41us) server_negotiation.cc:407] Connection header received
0925 07:27:58.153583 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:58.153585 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:58.153601 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:58.153674 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:27:58.153938 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:58.155036 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:58.156331 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:58.157165 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:58.157216 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:58.157262 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:58.157577 (+ 315us) server_negotiation.cc:299] Negotiation successful
0925 07:27:58.157613 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":76,"threads_started":1}
I20260925 07:27:58.157745 19781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f82d9531baa24f5ea83e65a0b12c4c71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:27:58.158282 19647 tablet_service.cc:1467] Processing CreateTablet for tablet f82d9531baa24f5ea83e65a0b12c4c71 (DEFAULT_TABLE table=test-table [id=e5f5618858f14d2f89eda69c6d461c12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:27:58.158501 19647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f82d9531baa24f5ea83e65a0b12c4c71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:27:58.160220 19995 tablet_bootstrap.cc:492] T f82d9531baa24f5ea83e65a0b12c4c71 P 7cce4a7fbbd34fbd86be2ab1c4671796: Bootstrap starting.
I20260925 07:27:58.160800 19996 tablet_bootstrap.cc:492] T f82d9531baa24f5ea83e65a0b12c4c71 P de941b832d3d46fc82275294327a08ac: Bootstrap starting.
I20260925 07:27:58.160943 19915 tablet_service.cc:1467] Processing CreateTablet for tablet f82d9531baa24f5ea83e65a0b12c4c71 (DEFAULT_TABLE table=test-table [id=e5f5618858f14d2f89eda69c6d461c12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:27:58.161280 19915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f82d9531baa24f5ea83e65a0b12c4c71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:27:58.161345 19995 tablet_bootstrap.cc:654] T f82d9531baa24f5ea83e65a0b12c4c71 P 7cce4a7fbbd34fbd86be2ab1c4671796: Neither blocks nor log segments found. Creating new log.
I20260925 07:27:58.161504 19996 tablet_bootstrap.cc:654] T f82d9531baa24f5ea83e65a0b12c4c71 P de941b832d3d46fc82275294327a08ac: Neither blocks nor log segments found. Creating new log.
I20260925 07:27:58.161964 19995 log.cc:824] T f82d9531baa24f5ea83e65a0b12c4c71 P 7cce4a7fbbd34fbd86be2ab1c4671796: Log is configured to *not* fsync() on all Append() calls
I20260925 07:27:58.162144 19996 log.cc:824] T f82d9531baa24f5ea83e65a0b12c4c71 P de941b832d3d46fc82275294327a08ac: Log is configured to *not* fsync() on all Append() calls
I20260925 07:27:58.162739 19995 tablet_bootstrap.cc:492] T f82d9531baa24f5ea83e65a0b12c4c71 P 7cce4a7fbbd34fbd86be2ab1c4671796: No bootstrap required, opened a new log
I20260925 07:27:58.162791 19995 ts_tablet_manager.cc:1397] T f82d9531baa24f5ea83e65a0b12c4c71 P 7cce4a7fbbd34fbd86be2ab1c4671796: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260925 07:27:58.162959 19996 tablet_bootstrap.cc:492] T f82d9531baa24f5ea83e65a0b12c4c71 P de941b832d3d46fc82275294327a08ac: No bootstrap required, opened a new log
I20260925 07:27:58.163055 19996 ts_tablet_manager.cc:1397] T f82d9531baa24f5ea83e65a0b12c4c71 P de941b832d3d46fc82275294327a08ac: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 07:27:58.164294 19999 tablet_bootstrap.cc:492] T f82d9531baa24f5ea83e65a0b12c4c71 P 308d38b0e2fd42d7afe17e8340b87f16: Bootstrap starting.
I20260925 07:27:58.164255 19995 raft_consensus.cc:348] T f82d9531baa24f5ea83e65a0b12c4c71 P 7cce4a7fbbd34fbd86be2ab1c4671796 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308d38b0e2fd42d7afe17e8340b87f16" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 35777 } } peers { permanent_uuid: "de941b832d3d46fc82275294327a08ac" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 35035 } } peers { permanent_uuid: "7cce4a7fbbd34fbd86be2ab1c4671796" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 38307 } }
I20260925 07:27:58.164419 19995 raft_consensus.cc:374] T f82d9531baa24f5ea83e65a0b12c4c71 P 7cce4a7fbbd34fbd86be2ab1c4671796 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:27:58.164465 19995 raft_consensus.cc:729] T f82d9531baa24f5ea83e65a0b12c4c71 P 7cce4a7fbbd34fbd86be2ab1c4671796 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cce4a7fbbd34fbd86be2ab1c4671796, State: Initialized, Role: FOLLOWER
I20260925 07:27:58.164569 19995 consensus_queue.cc:260] T f82d9531baa24f5ea83e65a0b12c4c71 P 7cce4a7fbbd34fbd86be2ab1c4671796 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308d38b0e2fd42d7afe17e8340b87f16" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 35777 } } peers { permanent_uuid: "de941b832d3d46fc82275294327a08ac" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 35035 } } peers { permanent_uuid: "7cce4a7fbbd34fbd86be2ab1c4671796" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 38307 } }
I20260925 07:27:58.164873 19995 ts_tablet_manager.cc:1428] T f82d9531baa24f5ea83e65a0b12c4c71 P 7cce4a7fbbd34fbd86be2ab1c4671796: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260925 07:27:58.165009 19847 heartbeater.cc:503] Master 127.5.158.126:43241 was elected leader, sending a full tablet report...
I20260925 07:27:58.165171 19999 tablet_bootstrap.cc:654] T f82d9531baa24f5ea83e65a0b12c4c71 P 308d38b0e2fd42d7afe17e8340b87f16: Neither blocks nor log segments found. Creating new log.
I20260925 07:27:58.165349 19996 raft_consensus.cc:348] T f82d9531baa24f5ea83e65a0b12c4c71 P de941b832d3d46fc82275294327a08ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308d38b0e2fd42d7afe17e8340b87f16" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 35777 } } peers { permanent_uuid: "de941b832d3d46fc82275294327a08ac" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 35035 } } peers { permanent_uuid: "7cce4a7fbbd34fbd86be2ab1c4671796" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 38307 } }
I20260925 07:27:58.165506 19996 raft_consensus.cc:374] T f82d9531baa24f5ea83e65a0b12c4c71 P de941b832d3d46fc82275294327a08ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:27:58.165539 19996 raft_consensus.cc:729] T f82d9531baa24f5ea83e65a0b12c4c71 P de941b832d3d46fc82275294327a08ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de941b832d3d46fc82275294327a08ac, State: Initialized, Role: FOLLOWER
I20260925 07:27:58.165649 19996 consensus_queue.cc:260] T f82d9531baa24f5ea83e65a0b12c4c71 P de941b832d3d46fc82275294327a08ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308d38b0e2fd42d7afe17e8340b87f16" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 35777 } } peers { permanent_uuid: "de941b832d3d46fc82275294327a08ac" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 35035 } } peers { permanent_uuid: "7cce4a7fbbd34fbd86be2ab1c4671796" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 38307 } }
I20260925 07:27:58.165767 19999 log.cc:824] T f82d9531baa24f5ea83e65a0b12c4c71 P 308d38b0e2fd42d7afe17e8340b87f16: Log is configured to *not* fsync() on all Append() calls
I20260925 07:27:58.165906 19996 ts_tablet_manager.cc:1428] T f82d9531baa24f5ea83e65a0b12c4c71 P de941b832d3d46fc82275294327a08ac: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260925 07:27:58.166033 19713 heartbeater.cc:503] Master 127.5.158.126:43241 was elected leader, sending a full tablet report...
I20260925 07:27:58.166517 19999 tablet_bootstrap.cc:492] T f82d9531baa24f5ea83e65a0b12c4c71 P 308d38b0e2fd42d7afe17e8340b87f16: No bootstrap required, opened a new log
I20260925 07:27:58.166604 19999 ts_tablet_manager.cc:1397] T f82d9531baa24f5ea83e65a0b12c4c71 P 308d38b0e2fd42d7afe17e8340b87f16: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260925 07:27:58.168185 19999 raft_consensus.cc:348] T f82d9531baa24f5ea83e65a0b12c4c71 P 308d38b0e2fd42d7afe17e8340b87f16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308d38b0e2fd42d7afe17e8340b87f16" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 35777 } } peers { permanent_uuid: "de941b832d3d46fc82275294327a08ac" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 35035 } } peers { permanent_uuid: "7cce4a7fbbd34fbd86be2ab1c4671796" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 38307 } }
I20260925 07:27:58.168380 19999 raft_consensus.cc:374] T f82d9531baa24f5ea83e65a0b12c4c71 P 308d38b0e2fd42d7afe17e8340b87f16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:27:58.168423 19999 raft_consensus.cc:729] T f82d9531baa24f5ea83e65a0b12c4c71 P 308d38b0e2fd42d7afe17e8340b87f16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 308d38b0e2fd42d7afe17e8340b87f16, State: Initialized, Role: FOLLOWER
I20260925 07:27:58.168520 19999 consensus_queue.cc:260] T f82d9531baa24f5ea83e65a0b12c4c71 P 308d38b0e2fd42d7afe17e8340b87f16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308d38b0e2fd42d7afe17e8340b87f16" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 35777 } } peers { permanent_uuid: "de941b832d3d46fc82275294327a08ac" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 35035 } } peers { permanent_uuid: "7cce4a7fbbd34fbd86be2ab1c4671796" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 38307 } }
I20260925 07:27:58.168795 19999 ts_tablet_manager.cc:1428] T f82d9531baa24f5ea83e65a0b12c4c71 P 308d38b0e2fd42d7afe17e8340b87f16: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260925 07:27:58.168828 19981 heartbeater.cc:503] Master 127.5.158.126:43241 was elected leader, sending a full tablet report...
W20260925 07:27:58.231022 19848 tablet.cc:2369] T f82d9531baa24f5ea83e65a0b12c4c71 P 7cce4a7fbbd34fbd86be2ab1c4671796: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 07:27:58.344017 19714 tablet.cc:2369] T f82d9531baa24f5ea83e65a0b12c4c71 P de941b832d3d46fc82275294327a08ac: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 07:27:58.346557 20000 raft_consensus.cc:482] T f82d9531baa24f5ea83e65a0b12c4c71 P 7cce4a7fbbd34fbd86be2ab1c4671796 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 07:27:58.346647 20000 raft_consensus.cc:504] T f82d9531baa24f5ea83e65a0b12c4c71 P 7cce4a7fbbd34fbd86be2ab1c4671796 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308d38b0e2fd42d7afe17e8340b87f16" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 35777 } } peers { permanent_uuid: "de941b832d3d46fc82275294327a08ac" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 35035 } } peers { permanent_uuid: "7cce4a7fbbd34fbd86be2ab1c4671796" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 38307 } }
I20260925 07:27:58.346930 20000 leader_election.cc:288] T f82d9531baa24f5ea83e65a0b12c4c71 P 7cce4a7fbbd34fbd86be2ab1c4671796 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 308d38b0e2fd42d7afe17e8340b87f16 (127.5.158.67:35777), de941b832d3d46fc82275294327a08ac (127.5.158.65:35035)
I20260925 07:27:58.349910 19849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:58.347145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.67:35777 (local address 127.5.158.66:37993)
0925 07:27:58.347200 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:58.347210 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 07:27:58.347274 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:27:58.347451 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:27:58.347452 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:27:58.347454 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:27:58.347544 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:58.347546 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:58.348191 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:58.348192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:58.349035 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:58.349037 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:58.349638 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:58.349639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:58.349714 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:58.349718 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 07:27:58.349790 (+ 72us) client_negotiation.cc:240] Negotiation successful
0925 07:27:58.349828 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260925 07:27:58.349928 19993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:58.347206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:37993 (local address 127.5.158.67:35777)
0925 07:27:58.347265 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:58.347268 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:58.347277 (+ 9us) server_negotiation.cc:407] Connection header received
0925 07:27:58.347336 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:58.347337 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:58.347346 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:58.347419 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:27:58.347628 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:58.348116 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:58.349211 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:58.349560 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:58.349613 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:58.349619 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:58.349799 (+ 180us) server_negotiation.cc:299] Negotiation successful
0925 07:27:58.349860 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260925 07:27:58.350005 20004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:58.347270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:35035 (local address 127.5.158.66:43383)
0925 07:27:58.347429 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:58.347438 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 07:27:58.347492 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:27:58.347626 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:27:58.347627 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:27:58.347628 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:27:58.347692 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:58.347693 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:58.348313 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:58.348314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:58.349044 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:58.349046 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:58.349692 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:58.349692 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:58.349816 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:58.349821 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 07:27:58.349898 (+ 77us) client_negotiation.cc:240] Negotiation successful
0925 07:27:58.349930 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":7,"thread_start_us":98,"threads_started":1}
I20260925 07:27:58.350052 19994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:58.347315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:43383 (local address 127.5.158.65:35035)
0925 07:27:58.347361 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:58.347363 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:58.347501 (+ 138us) server_negotiation.cc:407] Connection header received
0925 07:27:58.347551 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:58.347552 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:58.347562 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:58.347614 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:27:58.347756 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:58.348233 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:58.349171 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:58.349582 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:58.349678 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:58.349686 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:58.349913 (+ 227us) server_negotiation.cc:299] Negotiation successful
0925 07:27:58.349975 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 07:27:58.350324 19935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f82d9531baa24f5ea83e65a0b12c4c71" candidate_uuid: "7cce4a7fbbd34fbd86be2ab1c4671796" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "308d38b0e2fd42d7afe17e8340b87f16" is_pre_election: true
I20260925 07:27:58.350340 19667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f82d9531baa24f5ea83e65a0b12c4c71" candidate_uuid: "7cce4a7fbbd34fbd86be2ab1c4671796" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de941b832d3d46fc82275294327a08ac" is_pre_election: true
I20260925 07:27:58.350461 19667 raft_consensus.cc:2454] T f82d9531baa24f5ea83e65a0b12c4c71 P de941b832d3d46fc82275294327a08ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cce4a7fbbd34fbd86be2ab1c4671796 in term 0.
I20260925 07:27:58.350478 19935 raft_consensus.cc:2454] T f82d9531baa24f5ea83e65a0b12c4c71 P 308d38b0e2fd42d7afe17e8340b87f16 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cce4a7fbbd34fbd86be2ab1c4671796 in term 0.
I20260925 07:27:58.350644 19736 leader_election.cc:302] T f82d9531baa24f5ea83e65a0b12c4c71 P 7cce4a7fbbd34fbd86be2ab1c4671796 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cce4a7fbbd34fbd86be2ab1c4671796, de941b832d3d46fc82275294327a08ac; no voters:
I20260925 07:27:58.350802 20000 raft_consensus.cc:2781] T f82d9531baa24f5ea83e65a0b12c4c71 P 7cce4a7fbbd34fbd86be2ab1c4671796 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 07:27:58.350891 20000 raft_consensus.cc:482] T f82d9531baa24f5ea83e65a0b12c4c71 P 7cce4a7fbbd34fbd86be2ab1c4671796 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 07:27:58.350929 20000 raft_consensus.cc:3037] T f82d9531baa24f5ea83e65a0b12c4c71 P 7cce4a7fbbd34fbd86be2ab1c4671796 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:27:58.351629 20000 raft_consensus.cc:504] T f82d9531baa24f5ea83e65a0b12c4c71 P 7cce4a7fbbd34fbd86be2ab1c4671796 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308d38b0e2fd42d7afe17e8340b87f16" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 35777 } } peers { permanent_uuid: "de941b832d3d46fc82275294327a08ac" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 35035 } } peers { permanent_uuid: "7cce4a7fbbd34fbd86be2ab1c4671796" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 38307 } }
I20260925 07:27:58.351765 20000 leader_election.cc:288] T f82d9531baa24f5ea83e65a0b12c4c71 P 7cce4a7fbbd34fbd86be2ab1c4671796 [CANDIDATE]: Term 1 election: Requested vote from peers 308d38b0e2fd42d7afe17e8340b87f16 (127.5.158.67:35777), de941b832d3d46fc82275294327a08ac (127.5.158.65:35035)
I20260925 07:27:58.351949 19667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f82d9531baa24f5ea83e65a0b12c4c71" candidate_uuid: "7cce4a7fbbd34fbd86be2ab1c4671796" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de941b832d3d46fc82275294327a08ac"
I20260925 07:27:58.351950 19935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f82d9531baa24f5ea83e65a0b12c4c71" candidate_uuid: "7cce4a7fbbd34fbd86be2ab1c4671796" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "308d38b0e2fd42d7afe17e8340b87f16"
I20260925 07:27:58.352025 19667 raft_consensus.cc:3037] T f82d9531baa24f5ea83e65a0b12c4c71 P de941b832d3d46fc82275294327a08ac [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:27:58.352047 19935 raft_consensus.cc:3037] T f82d9531baa24f5ea83e65a0b12c4c71 P 308d38b0e2fd42d7afe17e8340b87f16 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:27:58.352785 19667 raft_consensus.cc:2454] T f82d9531baa24f5ea83e65a0b12c4c71 P de941b832d3d46fc82275294327a08ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cce4a7fbbd34fbd86be2ab1c4671796 in term 1.
I20260925 07:27:58.352890 19935 raft_consensus.cc:2454] T f82d9531baa24f5ea83e65a0b12c4c71 P 308d38b0e2fd42d7afe17e8340b87f16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cce4a7fbbd34fbd86be2ab1c4671796 in term 1.
I20260925 07:27:58.352968 19736 leader_election.cc:302] T f82d9531baa24f5ea83e65a0b12c4c71 P 7cce4a7fbbd34fbd86be2ab1c4671796 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cce4a7fbbd34fbd86be2ab1c4671796, de941b832d3d46fc82275294327a08ac; no voters:
I20260925 07:27:58.353103 20000 raft_consensus.cc:2781] T f82d9531baa24f5ea83e65a0b12c4c71 P 7cce4a7fbbd34fbd86be2ab1c4671796 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:27:58.353426 20000 raft_consensus.cc:686] T f82d9531baa24f5ea83e65a0b12c4c71 P 7cce4a7fbbd34fbd86be2ab1c4671796 [term 1 LEADER]: Becoming Leader. State: Replica: 7cce4a7fbbd34fbd86be2ab1c4671796, State: Running, Role: LEADER
I20260925 07:27:58.353542 20000 consensus_queue.cc:237] T f82d9531baa24f5ea83e65a0b12c4c71 P 7cce4a7fbbd34fbd86be2ab1c4671796 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308d38b0e2fd42d7afe17e8340b87f16" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 35777 } } peers { permanent_uuid: "de941b832d3d46fc82275294327a08ac" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 35035 } } peers { permanent_uuid: "7cce4a7fbbd34fbd86be2ab1c4671796" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 38307 } }
I20260925 07:27:58.354308 19522 catalog_manager.cc:5495] T f82d9531baa24f5ea83e65a0b12c4c71 P 7cce4a7fbbd34fbd86be2ab1c4671796 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cce4a7fbbd34fbd86be2ab1c4671796 (127.5.158.66). New cstate: current_term: 1 leader_uuid: "7cce4a7fbbd34fbd86be2ab1c4671796" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308d38b0e2fd42d7afe17e8340b87f16" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 35777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de941b832d3d46fc82275294327a08ac" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 35035 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cce4a7fbbd34fbd86be2ab1c4671796" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 38307 } health_report { overall_health: HEALTHY } } }
W20260925 07:27:58.372393 19982 tablet.cc:2369] T f82d9531baa24f5ea83e65a0b12c4c71 P 308d38b0e2fd42d7afe17e8340b87f16: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 07:27:58.385046 19991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:58.382787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54592 (local address 127.5.158.66:38307)
0925 07:27:58.382842 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:58.382844 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:58.382858 (+ 14us) server_negotiation.cc:407] Connection header received
0925 07:27:58.382905 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:58.382906 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:58.382920 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:58.382977 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 07:27:58.383108 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:58.383519 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:58.384262 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:58.384458 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:58.384533 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:58.384704 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 07:27:58.384797 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 07:27:58.384888 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:58.384954 (+ 66us) server_negotiation.cc:299] Negotiation successful
0925 07:27:58.384969 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 07:27:58.386524 19935 raft_consensus.cc:1261] T f82d9531baa24f5ea83e65a0b12c4c71 P 308d38b0e2fd42d7afe17e8340b87f16 [term 1 FOLLOWER]: Refusing update from remote peer 7cce4a7fbbd34fbd86be2ab1c4671796: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:27:58.386556 19667 raft_consensus.cc:1261] T f82d9531baa24f5ea83e65a0b12c4c71 P de941b832d3d46fc82275294327a08ac [term 1 FOLLOWER]: Refusing update from remote peer 7cce4a7fbbd34fbd86be2ab1c4671796: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:27:58.386822 20000 consensus_queue.cc:1035] T f82d9531baa24f5ea83e65a0b12c4c71 P 7cce4a7fbbd34fbd86be2ab1c4671796 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de941b832d3d46fc82275294327a08ac" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 35035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:27:58.386965 20005 consensus_queue.cc:1035] T f82d9531baa24f5ea83e65a0b12c4c71 P 7cce4a7fbbd34fbd86be2ab1c4671796 [LEADER]: Connected to new peer: Peer: permanent_uuid: "308d38b0e2fd42d7afe17e8340b87f16" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 35777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:27:58.389273 20009 mvcc.cc:205] Tried to move back new op lower bound from 7333155956269154304 to 7333155956136673280. Current Snapshot: MvccSnapshot[applied={T|T < 7333155956269154304}]
I20260925 07:27:58.390920 19521 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36142:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 07:27:58.390996 19521 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36142:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 07:27:58.391716 19521 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9: Sending DeleteTablet for 3 replicas of tablet f82d9531baa24f5ea83e65a0b12c4c71
I20260925 07:27:58.392041 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 19584
I20260925 07:27:58.392057 19781 tablet_service.cc:1514] Processing DeleteTablet for tablet f82d9531baa24f5ea83e65a0b12c4c71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:27:58 UTC) from {username='kudu', principal='kudu/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:54578
I20260925 07:27:58.392067 19915 tablet_service.cc:1514] Processing DeleteTablet for tablet f82d9531baa24f5ea83e65a0b12c4c71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:27:58 UTC) from {username='kudu', principal='kudu/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:46690
I20260925 07:27:58.392599 20018 tablet_replica.cc:330] T f82d9531baa24f5ea83e65a0b12c4c71 P 308d38b0e2fd42d7afe17e8340b87f16: stopping tablet replica
I20260925 07:27:58.392678 20018 raft_consensus.cc:2229] T f82d9531baa24f5ea83e65a0b12c4c71 P 308d38b0e2fd42d7afe17e8340b87f16 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 07:27:58.392758 20018 raft_consensus.cc:2258] T f82d9531baa24f5ea83e65a0b12c4c71 P 308d38b0e2fd42d7afe17e8340b87f16 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:27:58.392911 20017 tablet_replica.cc:330] T f82d9531baa24f5ea83e65a0b12c4c71 P 7cce4a7fbbd34fbd86be2ab1c4671796: stopping tablet replica
I20260925 07:27:58.392978 20017 raft_consensus.cc:2229] T f82d9531baa24f5ea83e65a0b12c4c71 P 7cce4a7fbbd34fbd86be2ab1c4671796 [term 1 LEADER]: Raft consensus shutting down.
I20260925 07:27:58.393049 20017 raft_consensus.cc:2258] T f82d9531baa24f5ea83e65a0b12c4c71 P 7cce4a7fbbd34fbd86be2ab1c4671796 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:27:58.393392 20018 ts_tablet_manager.cc:1905] T f82d9531baa24f5ea83e65a0b12c4c71 P 308d38b0e2fd42d7afe17e8340b87f16: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:27:58.394805 20017 ts_tablet_manager.cc:1905] T f82d9531baa24f5ea83e65a0b12c4c71 P 7cce4a7fbbd34fbd86be2ab1c4671796: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:27:58.395027 20018 ts_tablet_manager.cc:1918] T f82d9531baa24f5ea83e65a0b12c4c71 P 308d38b0e2fd42d7afe17e8340b87f16: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 07:27:58.395105 20018 log.cc:1196] T f82d9531baa24f5ea83e65a0b12c4c71 P 308d38b0e2fd42d7afe17e8340b87f16: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-2/wal/wals/f82d9531baa24f5ea83e65a0b12c4c71
I20260925 07:27:58.395367 20018 ts_tablet_manager.cc:1939] T f82d9531baa24f5ea83e65a0b12c4c71 P 308d38b0e2fd42d7afe17e8340b87f16: Deleting consensus metadata
I20260925 07:27:58.395751 19508 catalog_manager.cc:4841] TS 308d38b0e2fd42d7afe17e8340b87f16 (127.5.158.67:35777): tablet f82d9531baa24f5ea83e65a0b12c4c71 (table test-table [id=e5f5618858f14d2f89eda69c6d461c12]) successfully deleted
I20260925 07:27:58.396467 20017 ts_tablet_manager.cc:1918] T f82d9531baa24f5ea83e65a0b12c4c71 P 7cce4a7fbbd34fbd86be2ab1c4671796: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 07:27:58.396517 20017 log.cc:1196] T f82d9531baa24f5ea83e65a0b12c4c71 P 7cce4a7fbbd34fbd86be2ab1c4671796: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-1/wal/wals/f82d9531baa24f5ea83e65a0b12c4c71
I20260925 07:27:58.396836 20017 ts_tablet_manager.cc:1939] T f82d9531baa24f5ea83e65a0b12c4c71 P 7cce4a7fbbd34fbd86be2ab1c4671796: Deleting consensus metadata
I20260925 07:27:58.397182 19507 catalog_manager.cc:4841] TS 7cce4a7fbbd34fbd86be2ab1c4671796 (127.5.158.66:38307): tablet f82d9531baa24f5ea83e65a0b12c4c71 (table test-table [id=e5f5618858f14d2f89eda69c6d461c12]) successfully deleted
W20260925 07:27:58.397394 19508 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.158.65:35035 (error 108)
I20260925 07:27:58.397783 19989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:58.397655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:35035 (local address 127.0.0.1:60844)
0925 07:27:58.397713 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:58.397754 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.65:35035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260925 07:27:58.397907 19508 catalog_manager.cc:4584] TS de941b832d3d46fc82275294327a08ac (127.5.158.65:35035): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f82d9531baa24f5ea83e65a0b12c4c71: Network error: Client connection negotiation failed: client connection to 127.5.158.65:35035: connect: Connection refused (error 111)
I20260925 07:27:58.401229 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 19718
I20260925 07:27:58.406661 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 19852
I20260925 07:27:58.410449 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 19490
I20260925 07:27:58.415197 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:43241
--webserver_interface=127.5.158.126
--webserver_port=43063
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:42351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.158.126:43241
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:27:58.489739 20020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:27:58.489952 20020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:27:58.489991 20020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:27:58.491030 20020 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:27:58.491147 20020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:27:58.491182 20020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:27:58.491214 20020 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:27:58.491240 20020 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:27:58.492571 20020 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:42351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.5.158.126:43241
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:43241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.158.126
--webserver_port=43063
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20020
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:27:58.492823 20020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:27:58.493057 20020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:27:58.500912 20026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:27:58.500893 20029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:27:58.501070 20020 server_base.cc:1034] running on GCE node
W20260925 07:27:58.500893 20027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:27:58.501436 20020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:27:58.501747 20020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:27:58.502911 20020 hybrid_clock.cc:648] HybridClock initialized: now 1790321278502888 us; error 34 us; skew 500 ppm
Sep 25 07:27:58 dist-test-slave-1drq krb5kdc[19477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321278, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:27:58.505326 20020 init.cc:377] Logged in from keytab as kudu/127.5.158.126@KRBTEST.COM (short username kudu)
I20260925 07:27:58.505759 20020 webserver.cc:460] Webserver started at http://127.5.158.126:43063/ using document root <none> and password file <none>
I20260925 07:27:58.506003 20020 fs_manager.cc:359] Metadata directory not provided
I20260925 07:27:58.506098 20020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:27:58.507344 20020 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 07:27:58.508039 20036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:58.508217 20020 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:27:58.508330 20020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "e8e5d0766eb541f5b6c08f1844b6d6e9"
format_stamp: "Formatted at 2026-09-25 07:27:57 on dist-test-slave-1drq"
server_key: "489dfa6637e0085f04459f57da95f3a2"
server_key_iv: "05622286d32ccad4a4b13f49ad505c53"
server_key_version: "encryptionkey@0"
I20260925 07:27:58.508800 20020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:27:58.526219 20020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:27:58.527720 20020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:27:58.527818 20020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:27:58.531862 20020 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.126:43241
I20260925 07:27:58.531890 20088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.126:43241 every 8 connection(s)
I20260925 07:27:58.532267 20020 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:27:58.533751 20089 sys_catalog.cc:263] Verifying existing consensus state
I20260925 07:27:58.534365 20089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9: Bootstrap starting.
I20260925 07:27:58.539414 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 20020
I20260925 07:27:58.539721 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.65:35035
--local_ip_for_outbound_sockets=127.5.158.65
--tserver_master_addrs=127.5.158.126:43241
--webserver_port=35217
--webserver_interface=127.5.158.65
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab
--principal=kudu/127.5.158.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:42351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 07:27:58.542039 20089 log.cc:824] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9: Log is configured to *not* fsync() on all Append() calls
I20260925 07:27:58.543810 20089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 07:27:58.544092 20089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9: Bootstrap complete.
I20260925 07:27:58.546188 20089 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8e5d0766eb541f5b6c08f1844b6d6e9" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 43241 } }
I20260925 07:27:58.546398 20089 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8e5d0766eb541f5b6c08f1844b6d6e9, State: Initialized, Role: FOLLOWER
I20260925 07:27:58.546573 20089 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8e5d0766eb541f5b6c08f1844b6d6e9" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 43241 } }
I20260925 07:27:58.546657 20089 raft_consensus.cc:388] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 07:27:58.546706 20089 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 07:27:58.546747 20089 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9 [term 1 FOLLOWER]: Advancing to term 2
I20260925 07:27:58.547632 20089 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8e5d0766eb541f5b6c08f1844b6d6e9" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 43241 } }
I20260925 07:27:58.547741 20089 leader_election.cc:302] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e8e5d0766eb541f5b6c08f1844b6d6e9; no voters:
I20260925 07:27:58.547921 20089 leader_election.cc:288] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260925 07:27:58.547950 20093 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 07:27:58.548048 20093 raft_consensus.cc:686] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9 [term 2 LEADER]: Becoming Leader. State: Replica: e8e5d0766eb541f5b6c08f1844b6d6e9, State: Running, Role: LEADER
I20260925 07:27:58.548128 20093 consensus_queue.cc:237] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8e5d0766eb541f5b6c08f1844b6d6e9" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 43241 } }
I20260925 07:27:58.548153 20089 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:27:58.548383 20095 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8e5d0766eb541f5b6c08f1844b6d6e9. Latest consensus state: current_term: 2 leader_uuid: "e8e5d0766eb541f5b6c08f1844b6d6e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8e5d0766eb541f5b6c08f1844b6d6e9" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 43241 } } }
I20260925 07:27:58.548487 20095 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9 [sys.catalog]: This master's current role is: LEADER
I20260925 07:27:58.548645 20094 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e8e5d0766eb541f5b6c08f1844b6d6e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8e5d0766eb541f5b6c08f1844b6d6e9" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 43241 } } }
I20260925 07:27:58.548702 20098 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 07:27:58.548717 20094 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9 [sys.catalog]: This master's current role is: LEADER
I20260925 07:27:58.549571 20098 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 07:27:58.549721 20098 catalog_manager.cc:1259] Loaded cluster ID: 6be6538bdebe4e52a9afcf87a34de8dd
I20260925 07:27:58.549791 20098 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 07:27:58.550215 20098 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260925 07:27:58.554361 20098 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260925 07:27:58.556638 20098 catalog_manager.cc:1504] Loading token signing keys...
I20260925 07:27:58.556802 20098 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9: Loaded TSK: 0
I20260925 07:27:58.556983 20098 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 07:27:58.615044 20091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:27:58.615227 20091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:27:58.615248 20091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:27:58.616230 20091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:27:58.616268 20091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:27:58.616292 20091 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.65
I20260925 07:27:58.617573 20091 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:42351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab
--principal=kudu/127.5.158.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.65:35035
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.158.65
--webserver_port=35217
--tserver_master_addrs=127.5.158.126:43241
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20091
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.65
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:27:58.617779 20091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:27:58.617995 20091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:27:58.622327 20117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:27:58.622370 20116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:27:58.622615 20091 server_base.cc:1034] running on GCE node
W20260925 07:27:58.622661 20119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:27:58.622867 20091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:27:58.623129 20091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:27:58.624282 20091 hybrid_clock.cc:648] HybridClock initialized: now 1790321278624272 us; error 27 us; skew 500 ppm
Sep 25 07:27:58 dist-test-slave-1drq krb5kdc[19477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321278, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:27:58.626761 20091 init.cc:377] Logged in from keytab as kudu/127.5.158.65@KRBTEST.COM (short username kudu)
I20260925 07:27:58.627135 20091 webserver.cc:460] Webserver started at http://127.5.158.65:35217/ using document root <none> and password file <none>
I20260925 07:27:58.627321 20091 fs_manager.cc:359] Metadata directory not provided
I20260925 07:27:58.627414 20091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:27:58.628793 20091 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 07:27:58.629537 20126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:58.629701 20091 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:58.629787 20091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-0/wal
uuid: "de941b832d3d46fc82275294327a08ac"
format_stamp: "Formatted at 2026-09-25 07:27:57 on dist-test-slave-1drq"
server_key: "cf9e5c9c55e1d3ec00bba553e70b789d"
server_key_iv: "0eb4622a47088c9d8b19c21bf82e4729"
server_key_version: "encryptionkey@0"
I20260925 07:27:58.630055 20091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:27:58.666692 20091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:27:58.668272 20091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:27:58.668409 20091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:27:58.668612 20091 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:27:58.668965 20133 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260925 07:27:58.670171 20091 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260925 07:27:58.670228 20091 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20260925 07:27:58.670269 20091 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260925 07:27:58.670791 20091 ts_tablet_manager.cc:610] Registered 1 tablets
I20260925 07:27:58.670840 20091 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260925 07:27:58.670887 20133 tablet_bootstrap.cc:492] T f82d9531baa24f5ea83e65a0b12c4c71 P de941b832d3d46fc82275294327a08ac: Bootstrap starting.
I20260925 07:27:58.677045 20091 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.65:35035
I20260925 07:27:58.677155 20240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.65:35035 every 8 connection(s)
I20260925 07:27:58.677404 20091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
I20260925 07:27:58.679082 20133 log.cc:824] T f82d9531baa24f5ea83e65a0b12c4c71 P de941b832d3d46fc82275294327a08ac: Log is configured to *not* fsync() on all Append() calls
I20260925 07:27:58.681275 20133 tablet_bootstrap.cc:492] T f82d9531baa24f5ea83e65a0b12c4c71 P de941b832d3d46fc82275294327a08ac: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 07:27:58.681537 20133 tablet_bootstrap.cc:492] T f82d9531baa24f5ea83e65a0b12c4c71 P de941b832d3d46fc82275294327a08ac: Bootstrap complete.
I20260925 07:27:58.682610 20133 ts_tablet_manager.cc:1397] T f82d9531baa24f5ea83e65a0b12c4c71 P de941b832d3d46fc82275294327a08ac: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.010s
I20260925 07:27:58.683595 20133 raft_consensus.cc:348] T f82d9531baa24f5ea83e65a0b12c4c71 P de941b832d3d46fc82275294327a08ac [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308d38b0e2fd42d7afe17e8340b87f16" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 35777 } } peers { permanent_uuid: "de941b832d3d46fc82275294327a08ac" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 35035 } } peers { permanent_uuid: "7cce4a7fbbd34fbd86be2ab1c4671796" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 38307 } }
I20260925 07:27:58.683929 20133 raft_consensus.cc:729] T f82d9531baa24f5ea83e65a0b12c4c71 P de941b832d3d46fc82275294327a08ac [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: de941b832d3d46fc82275294327a08ac, State: Initialized, Role: FOLLOWER
I20260925 07:27:58.684053 20133 consensus_queue.cc:260] T f82d9531baa24f5ea83e65a0b12c4c71 P de941b832d3d46fc82275294327a08ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308d38b0e2fd42d7afe17e8340b87f16" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 35777 } } peers { permanent_uuid: "de941b832d3d46fc82275294327a08ac" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 35035 } } peers { permanent_uuid: "7cce4a7fbbd34fbd86be2ab1c4671796" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 38307 } }
Sep 25 07:27:58 dist-test-slave-1drq krb5kdc[19477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321278, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.65@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:27:58.684301 20133 ts_tablet_manager.cc:1428] T f82d9531baa24f5ea83e65a0b12c4c71 P de941b832d3d46fc82275294327a08ac: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260925 07:27:58.684762 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 20091
I20260925 07:27:58.685076 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.66:38307
--local_ip_for_outbound_sockets=127.5.158.66
--tserver_master_addrs=127.5.158.126:43241
--webserver_port=42683
--webserver_interface=127.5.158.66
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab
--principal=kudu/127.5.158.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:42351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 07:27:58.688014 20243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:58.678033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.65:36307)
0925 07:27:58.678203 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:58.678237 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 07:27:58.678594 (+ 357us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:27:58.679005 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:27:58.679008 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:27:58.679284 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:27:58.679439 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:58.679442 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:58.680845 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:58.680846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:58.681819 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:58.681821 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:58.682590 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:58.682590 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:58.682706 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:58.683117 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:27:58.683129 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:27:58.684556 (+ 1427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:27:58.686659 (+ 2103us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:27:58.686665 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:27:58.686667 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:27:58.686942 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:27:58.687173 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:27:58.687174 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:27:58.687174 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:27:58.687290 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:27:58.687649 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:27:58.687653 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:27:58.687741 (+ 88us) client_negotiation.cc:769] Sending connection context
0925 07:27:58.687812 (+ 71us) client_negotiation.cc:240] Negotiation successful
0925 07:27:58.687906 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":103,"threads_started":1}
I20260925 07:27:58.688195 20244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:58.678123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:36307 (local address 127.5.158.126:43241)
0925 07:27:58.678323 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:58.678334 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:58.678373 (+ 39us) server_negotiation.cc:407] Connection header received
0925 07:27:58.678782 (+ 409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:58.678789 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:58.678841 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:58.678988 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:27:58.679523 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:58.680735 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:58.681896 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:58.682514 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:58.682616 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:58.684657 (+ 2041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:27:58.684662 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:27:58.684662 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:27:58.684676 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:27:58.686570 (+ 1894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:58.687022 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:58.687024 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:58.687024 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:58.687097 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:58.687355 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:58.687355 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:58.687356 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:58.687588 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:27:58.687659 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:58.687884 (+ 225us) server_negotiation.cc:299] Negotiation successful
0925 07:27:58.688005 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":86,"threads_started":1}
I20260925 07:27:58.688697 20241 heartbeater.cc:347] Connected to a master server at 127.5.158.126:43241
I20260925 07:27:58.688808 20241 heartbeater.cc:464] Registering TS with master...
I20260925 07:27:58.688973 20241 heartbeater.cc:511] Master 127.5.158.126:43241 requested a full tablet report, sending...
I20260925 07:27:58.689640 20053 ts_manager.cc:194] Registered new tserver with Master: de941b832d3d46fc82275294327a08ac (127.5.158.65:35035)
I20260925 07:27:58.690639 20053 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.65@KRBTEST.COM'} at 127.5.158.65:36307
I20260925 07:27:58.691425 20241 heartbeater.cc:503] Master 127.5.158.126:43241 was elected leader, sending a full tablet report...
Sep 25 07:27:58 dist-test-slave-1drq krb5kdc[19477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321278, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.126@KRBTEST.COM for kudu/127.5.158.65@KRBTEST.COM
I20260925 07:27:58.698107 20248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:58.690161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:35035 (local address 127.0.0.1:60858)
0925 07:27:58.690355 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:58.690387 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0925 07:27:58.690447 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:27:58.691130 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:27:58.691135 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:27:58.691452 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:27:58.691551 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:58.691554 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:58.692381 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:58.692382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:58.693342 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:58.693345 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:58.694005 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:58.694006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:58.694064 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:58.694534 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:27:58.694539 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:27:58.695665 (+ 1126us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:27:58.697213 (+ 1548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:27:58.697218 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:27:58.697220 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:27:58.697377 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:27:58.697575 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:27:58.697576 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:27:58.697577 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:27:58.697645 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:27:58.697894 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:27:58.697898 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:27:58.697930 (+ 32us) client_negotiation.cc:769] Sending connection context
0925 07:27:58.697981 (+ 51us) client_negotiation.cc:240] Negotiation successful
0925 07:27:58.698053 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":93,"threads_started":1}
I20260925 07:27:58.698342 20249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:58.690210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60858 (local address 127.5.158.65:35035)
0925 07:27:58.690947 (+ 737us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:58.690950 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:58.690980 (+ 30us) server_negotiation.cc:407] Connection header received
0925 07:27:58.691009 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:58.691011 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:58.691024 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:58.691118 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:27:58.691625 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:58.692310 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:58.693435 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:58.693918 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:58.694005 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:58.695734 (+ 1729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:27:58.695740 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:27:58.695740 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:27:58.695746 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:27:58.697148 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:58.697443 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:58.697445 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:58.697445 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:58.697511 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:58.697699 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:58.697700 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:58.697700 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:58.697836 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:27:58.697900 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:58.698023 (+ 123us) server_negotiation.cc:299] Negotiation successful
0925 07:27:58.698101 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":713,"thread_start_us":83,"threads_started":1}
I20260925 07:27:58.698678 20175 tablet_service.cc:1514] Processing DeleteTablet for tablet f82d9531baa24f5ea83e65a0b12c4c71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:27:58 UTC) from {username='kudu', principal='kudu/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:60858
I20260925 07:27:58.699092 20250 tablet_replica.cc:330] stopping tablet replica
I20260925 07:27:58.699218 20250 raft_consensus.cc:2229] T f82d9531baa24f5ea83e65a0b12c4c71 P de941b832d3d46fc82275294327a08ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 07:27:58.699280 20250 raft_consensus.cc:2258] T f82d9531baa24f5ea83e65a0b12c4c71 P de941b832d3d46fc82275294327a08ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:27:58.699529 20250 ts_tablet_manager.cc:1905] T f82d9531baa24f5ea83e65a0b12c4c71 P de941b832d3d46fc82275294327a08ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:27:58.701422 20250 ts_tablet_manager.cc:1918] T f82d9531baa24f5ea83e65a0b12c4c71 P de941b832d3d46fc82275294327a08ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 07:27:58.701494 20250 log.cc:1196] T f82d9531baa24f5ea83e65a0b12c4c71 P de941b832d3d46fc82275294327a08ac: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-0/wal/wals/f82d9531baa24f5ea83e65a0b12c4c71
I20260925 07:27:58.701750 20250 ts_tablet_manager.cc:1939] T f82d9531baa24f5ea83e65a0b12c4c71 P de941b832d3d46fc82275294327a08ac: Deleting consensus metadata
I20260925 07:27:58.702214 20039 catalog_manager.cc:4841] TS de941b832d3d46fc82275294327a08ac (127.5.158.65:35035): tablet f82d9531baa24f5ea83e65a0b12c4c71 (table test-table [id=e5f5618858f14d2f89eda69c6d461c12]) successfully deleted
W20260925 07:27:58.760649 20247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:27:58.760877 20247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:27:58.760926 20247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:27:58.762197 20247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:27:58.762253 20247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:27:58.762297 20247 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.66
I20260925 07:27:58.763528 20247 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:42351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab
--principal=kudu/127.5.158.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.66:38307
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.158.66
--webserver_port=42683
--tserver_master_addrs=127.5.158.126:43241
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20247
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.66
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:27:58.763736 20247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:27:58.763991 20247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:27:58.768348 20255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:27:58.768473 20256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:27:58.768356 20258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:27:58.768452 20247 server_base.cc:1034] running on GCE node
I20260925 07:27:58.768731 20247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:27:58.768890 20247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:27:58.770025 20247 hybrid_clock.cc:648] HybridClock initialized: now 1790321278770001 us; error 37 us; skew 500 ppm
Sep 25 07:27:58 dist-test-slave-1drq krb5kdc[19477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321278, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:27:58.772145 20247 init.cc:377] Logged in from keytab as kudu/127.5.158.66@KRBTEST.COM (short username kudu)
I20260925 07:27:58.772441 20247 webserver.cc:460] Webserver started at http://127.5.158.66:42683/ using document root <none> and password file <none>
I20260925 07:27:58.772578 20247 fs_manager.cc:359] Metadata directory not provided
I20260925 07:27:58.772621 20247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:27:58.773729 20247 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260925 07:27:58.774260 20265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:58.774428 20247 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:58.774523 20247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-1/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-1/wal
uuid: "7cce4a7fbbd34fbd86be2ab1c4671796"
format_stamp: "Formatted at 2026-09-25 07:27:57 on dist-test-slave-1drq"
server_key: "b1699fda57649e2a8ab6e67583325ffc"
server_key_iv: "335357c6c2f5b66a3fb7cc7d82052fcf"
server_key_version: "encryptionkey@0"
I20260925 07:27:58.774811 20247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:27:58.780891 20247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:27:58.783031 20247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:27:58.783197 20247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:27:58.783392 20247 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:27:58.783648 20247 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:27:58.783700 20247 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:58.783742 20247 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:27:58.783771 20247 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:58.789644 20247 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.66:38307
I20260925 07:27:58.789703 20378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.66:38307 every 8 connection(s)
I20260925 07:27:58.790083 20247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
Sep 25 07:27:58 dist-test-slave-1drq krb5kdc[19477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321278, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.66@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:27:58.797912 20244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:58.790797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:45655 (local address 127.5.158.126:43241)
0925 07:27:58.790864 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:58.790866 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:58.791002 (+ 136us) server_negotiation.cc:407] Connection header received
0925 07:27:58.791456 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:58.791457 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:58.791472 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:58.791527 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:27:58.792299 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:58.792758 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:58.793711 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:58.793948 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:58.794034 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:58.795765 (+ 1731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:27:58.795767 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:27:58.795767 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:27:58.795773 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:27:58.796843 (+ 1070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:58.797217 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:58.797218 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:58.797218 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:58.797263 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:58.797462 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:58.797463 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:58.797463 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:58.797547 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:27:58.797591 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:58.797789 (+ 198us) server_negotiation.cc:299] Negotiation successful
0925 07:27:58.797859 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260925 07:27:58.798009 20381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:58.790756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.66:45655)
0925 07:27:58.790895 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:58.790931 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 07:27:58.791359 (+ 428us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:27:58.791587 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:27:58.791590 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:27:58.792052 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:27:58.792201 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:58.792204 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:58.792862 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:58.792865 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:58.793610 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:58.793612 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:58.794044 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:58.794045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:58.794159 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:58.794519 (+ 360us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:27:58.794525 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:27:58.795689 (+ 1164us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:27:58.796910 (+ 1221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:27:58.796915 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:27:58.796916 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:27:58.797161 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:27:58.797328 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:27:58.797329 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:27:58.797329 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:27:58.797412 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:27:58.797600 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:27:58.797604 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:27:58.797692 (+ 88us) client_negotiation.cc:769] Sending connection context
0925 07:27:58.797763 (+ 71us) client_negotiation.cc:240] Negotiation successful
0925 07:27:58.797866 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":78,"threads_started":1}
I20260925 07:27:58.798452 20379 heartbeater.cc:347] Connected to a master server at 127.5.158.126:43241
I20260925 07:27:58.798528 20379 heartbeater.cc:464] Registering TS with master...
I20260925 07:27:58.798629 20379 heartbeater.cc:511] Master 127.5.158.126:43241 requested a full tablet report, sending...
I20260925 07:27:58.798941 20053 ts_manager.cc:194] Registered new tserver with Master: 7cce4a7fbbd34fbd86be2ab1c4671796 (127.5.158.66:38307)
I20260925 07:27:58.799220 20053 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.66@KRBTEST.COM'} at 127.5.158.66:45655
I20260925 07:27:58.799896 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 20247
I20260925 07:27:58.800108 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.67:35777
--local_ip_for_outbound_sockets=127.5.158.67
--tserver_master_addrs=127.5.158.126:43241
--webserver_port=43081
--webserver_interface=127.5.158.67
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab
--principal=kudu/127.5.158.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:42351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:27:58.873358 20382 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:27:58.873508 20382 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:27:58.873528 20382 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:27:58.874641 20382 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:27:58.874681 20382 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:27:58.874706 20382 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.67
I20260925 07:27:58.875887 20382 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:42351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab
--principal=kudu/127.5.158.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.67:35777
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.158.67
--webserver_port=43081
--tserver_master_addrs=127.5.158.126:43241
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20382
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.67
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:27:58.876029 20382 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:27:58.876157 20382 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:27:58.880604 20387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:27:58.880628 20388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:27:58.880595 20390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:27:58.880620 20382 server_base.cc:1034] running on GCE node
I20260925 07:27:58.881001 20382 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:27:58.881274 20382 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:27:58.882453 20382 hybrid_clock.cc:648] HybridClock initialized: now 1790321278882442 us; error 24 us; skew 500 ppm
Sep 25 07:27:58 dist-test-slave-1drq krb5kdc[19477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321278, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:27:58.884609 20382 init.cc:377] Logged in from keytab as kudu/127.5.158.67@KRBTEST.COM (short username kudu)
I20260925 07:27:58.885025 20382 webserver.cc:460] Webserver started at http://127.5.158.67:43081/ using document root <none> and password file <none>
I20260925 07:27:58.885249 20382 fs_manager.cc:359] Metadata directory not provided
I20260925 07:27:58.885327 20382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:27:58.886449 20382 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 07:27:58.886967 20397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:58.887127 20382 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:27:58.887212 20382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-2/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-2/wal
uuid: "308d38b0e2fd42d7afe17e8340b87f16"
format_stamp: "Formatted at 2026-09-25 07:27:58 on dist-test-slave-1drq"
server_key: "21cc47f0abc344fa8483396a74e5c19b"
server_key_iv: "71f98021aca893db8c042fb45cc56cda"
server_key_version: "encryptionkey@0"
I20260925 07:27:58.887485 20382 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:27:58.893597 20382 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:27:58.895278 20382 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:27:58.895414 20382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:27:58.895591 20382 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:27:58.895840 20382 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:27:58.895891 20382 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:58.895925 20382 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:27:58.895949 20382 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:58.901124 20382 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.67:35777
I20260925 07:27:58.901165 20510 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.67:35777 every 8 connection(s)
I20260925 07:27:58.901472 20382 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
I20260925 07:27:58.904305 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 20382
Sep 25 07:27:58 dist-test-slave-1drq krb5kdc[19477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321278, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.67@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:27:58.911263 20515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:58.904665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36146 (local address 127.5.158.126:43241)
0925 07:27:58.905168 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:58.905170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:58.905176 (+ 6us) server_negotiation.cc:407] Connection header received
0925 07:27:58.905197 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:58.905198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:58.905208 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:58.905262 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:27:58.905858 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:58.906272 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:58.906899 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:58.907100 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:58.907173 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:58.908387 (+ 1214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:27:58.908388 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:27:58.908388 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:27:58.908395 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:27:58.910091 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:58.910409 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:58.910410 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:58.910410 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:58.910493 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:58.910723 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:58.910724 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:58.910724 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:58.910869 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:27:58.911027 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:58.911075 (+ 48us) server_negotiation.cc:299] Negotiation successful
0925 07:27:58.911155 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":49,"threads_started":1}
I20260925 07:27:58.911551 20244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:58.902136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:41811 (local address 127.5.158.126:43241)
0925 07:27:58.902213 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:58.902215 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:58.902340 (+ 125us) server_negotiation.cc:407] Connection header received
0925 07:27:58.902699 (+ 359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:58.902700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:58.902709 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:58.902754 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:27:58.903519 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:58.904007 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:58.905232 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:58.905515 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:58.905583 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:58.907285 (+ 1702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:27:58.907286 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:27:58.907287 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:27:58.907293 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:27:58.909414 (+ 2121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:58.909862 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:58.909863 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:58.909863 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:58.909917 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:27:58.910447 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:27:58.910448 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:27:58.910448 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:27:58.910558 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:27:58.910593 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:58.911378 (+ 785us) server_negotiation.cc:299] Negotiation successful
0925 07:27:58.911479 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260925 07:27:58.911847 20513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:58.902093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.67:41811)
0925 07:27:58.902240 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:58.902268 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0925 07:27:58.902613 (+ 345us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:27:58.902795 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:27:58.902798 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:27:58.903271 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:27:58.903439 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:58.903443 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:58.904094 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:58.904095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:58.905153 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:58.905156 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:58.905591 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:58.905592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:58.905675 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:58.906067 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:27:58.906074 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:27:58.907207 (+ 1133us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:27:58.909492 (+ 2285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:27:58.909497 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:27:58.909499 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:27:58.909753 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:27:58.909989 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:27:58.909990 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:27:58.909990 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:27:58.910109 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:27:58.910763 (+ 654us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:27:58.910778 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:27:58.911264 (+ 486us) client_negotiation.cc:769] Sending connection context
0925 07:27:58.911363 (+ 99us) client_negotiation.cc:240] Negotiation successful
0925 07:27:58.911579 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":88,"threads_started":1}
I20260925 07:27:58.912254 20511 heartbeater.cc:347] Connected to a master server at 127.5.158.126:43241
I20260925 07:27:58.912338 20511 heartbeater.cc:464] Registering TS with master...
I20260925 07:27:58.912489 20511 heartbeater.cc:511] Master 127.5.158.126:43241 requested a full tablet report, sending...
I20260925 07:27:58.912835 20053 ts_manager.cc:194] Registered new tserver with Master: 308d38b0e2fd42d7afe17e8340b87f16 (127.5.158.67:35777)
I20260925 07:27:58.913200 20053 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.67@KRBTEST.COM'} at 127.5.158.67:41811
I20260925 07:27:58.922845 5753 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 07:27:58.925468 20244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:58.923145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36158 (local address 127.5.158.126:43241)
0925 07:27:58.923198 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:58.923200 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:58.923246 (+ 46us) server_negotiation.cc:407] Connection header received
0925 07:27:58.923323 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:58.923323 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:58.923353 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:58.923439 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 07:27:58.923521 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:58.923959 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:58.924683 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:58.924882 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:58.924986 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:58.925205 (+ 219us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 07:27:58.925293 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 07:27:58.925373 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:58.925396 (+ 23us) server_negotiation.cc:299] Negotiation successful
0925 07:27:58.925409 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260925 07:27:58.925691 20053 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:36158:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 07:27:58.927079 20053 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 07:27:58.929214 20175 tablet_service.cc:1467] Processing CreateTablet for tablet faedc57931424efc8451ff0e05e1661b (DEFAULT_TABLE table=test-table [id=b451171f6f4943beae4a03fd43fb1223]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:27:58.929399 20175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet faedc57931424efc8451ff0e05e1661b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:27:58.930833 20133 tablet_bootstrap.cc:492] T faedc57931424efc8451ff0e05e1661b P de941b832d3d46fc82275294327a08ac: Bootstrap starting.
I20260925 07:27:58.931156 20133 tablet_bootstrap.cc:654] T faedc57931424efc8451ff0e05e1661b P de941b832d3d46fc82275294327a08ac: Neither blocks nor log segments found. Creating new log.
I20260925 07:27:58.932139 20133 tablet_bootstrap.cc:492] T faedc57931424efc8451ff0e05e1661b P de941b832d3d46fc82275294327a08ac: No bootstrap required, opened a new log
I20260925 07:27:58.932193 20133 ts_tablet_manager.cc:1397] T faedc57931424efc8451ff0e05e1661b P de941b832d3d46fc82275294327a08ac: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260925 07:27:58.932336 20248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:58.928655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.67:35777 (local address 127.0.0.1:46698)
0925 07:27:58.928743 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:58.928751 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 07:27:58.928779 (+ 28us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:27:58.929053 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:27:58.929054 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:27:58.929055 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:27:58.929105 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:58.929106 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:58.930166 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:58.930167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:58.931162 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:58.931164 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:58.932079 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:58.932080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:58.932148 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:58.932164 (+ 16us) client_negotiation.cc:769] Sending connection context
0925 07:27:58.932213 (+ 49us) client_negotiation.cc:240] Negotiation successful
0925 07:27:58.932259 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76,"mutex_wait_us":20}
I20260925 07:27:58.932353 20133 raft_consensus.cc:348] T faedc57931424efc8451ff0e05e1661b P de941b832d3d46fc82275294327a08ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308d38b0e2fd42d7afe17e8340b87f16" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 35777 } } peers { permanent_uuid: "7cce4a7fbbd34fbd86be2ab1c4671796" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 38307 } } peers { permanent_uuid: "de941b832d3d46fc82275294327a08ac" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 35035 } }
I20260925 07:27:58.932423 20133 raft_consensus.cc:374] T faedc57931424efc8451ff0e05e1661b P de941b832d3d46fc82275294327a08ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:27:58.932459 20133 raft_consensus.cc:729] T faedc57931424efc8451ff0e05e1661b P de941b832d3d46fc82275294327a08ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de941b832d3d46fc82275294327a08ac, State: Initialized, Role: FOLLOWER
I20260925 07:27:58.932466 20517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:58.928654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.66:38307 (local address 127.0.0.1:54602)
0925 07:27:58.928907 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:58.928917 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 07:27:58.928959 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:27:58.929253 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:27:58.929253 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:27:58.929255 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:27:58.929307 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:58.929309 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:58.930353 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:58.930353 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:58.931192 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:58.931195 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:58.932233 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:58.932234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:58.932300 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:58.932303 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 07:27:58.932352 (+ 49us) client_negotiation.cc:240] Negotiation successful
0925 07:27:58.932402 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"mutex_wait_us":16,"thread_start_us":61,"threads_started":1}
I20260925 07:27:58.932509 20133 consensus_queue.cc:260] T faedc57931424efc8451ff0e05e1661b P de941b832d3d46fc82275294327a08ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308d38b0e2fd42d7afe17e8340b87f16" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 35777 } } peers { permanent_uuid: "7cce4a7fbbd34fbd86be2ab1c4671796" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 38307 } } peers { permanent_uuid: "de941b832d3d46fc82275294327a08ac" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 35035 } }
I20260925 07:27:58.932566 20518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:58.928774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46698 (local address 127.5.158.67:35777)
0925 07:27:58.928907 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:58.928910 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:58.928917 (+ 7us) server_negotiation.cc:407] Connection header received
0925 07:27:58.928939 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:58.928941 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:58.928968 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:58.929052 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:27:58.929157 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:58.930071 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:58.931245 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:58.931996 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:58.932061 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:58.932120 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:58.932451 (+ 331us) server_negotiation.cc:299] Negotiation successful
0925 07:27:58.932496 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":69,"threads_started":1}
I20260925 07:27:58.932603 20133 ts_tablet_manager.cc:1428] T faedc57931424efc8451ff0e05e1661b P de941b832d3d46fc82275294327a08ac: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260925 07:27:58.932722 20519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:58.928865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54602 (local address 127.5.158.66:38307)
0925 07:27:58.929034 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:58.929038 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:58.929046 (+ 8us) server_negotiation.cc:407] Connection header received
0925 07:27:58.929080 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:58.929106 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:58.929122 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:58.929189 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:27:58.929381 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:58.930257 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:58.931300 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:58.932073 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:58.932140 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:58.932195 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:58.932605 (+ 410us) server_negotiation.cc:299] Negotiation successful
0925 07:27:58.932651 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":85,"threads_started":1}
I20260925 07:27:58.933120 20445 tablet_service.cc:1467] Processing CreateTablet for tablet faedc57931424efc8451ff0e05e1661b (DEFAULT_TABLE table=test-table [id=b451171f6f4943beae4a03fd43fb1223]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:27:58.933372 20445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet faedc57931424efc8451ff0e05e1661b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:27:58.933367 20300 tablet_service.cc:1467] Processing CreateTablet for tablet faedc57931424efc8451ff0e05e1661b (DEFAULT_TABLE table=test-table [id=b451171f6f4943beae4a03fd43fb1223]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:27:58.933606 20300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet faedc57931424efc8451ff0e05e1661b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:27:58.935868 20521 tablet_bootstrap.cc:492] T faedc57931424efc8451ff0e05e1661b P 308d38b0e2fd42d7afe17e8340b87f16: Bootstrap starting.
I20260925 07:27:58.936115 20522 tablet_bootstrap.cc:492] T faedc57931424efc8451ff0e05e1661b P 7cce4a7fbbd34fbd86be2ab1c4671796: Bootstrap starting.
I20260925 07:27:58.936558 20521 tablet_bootstrap.cc:654] T faedc57931424efc8451ff0e05e1661b P 308d38b0e2fd42d7afe17e8340b87f16: Neither blocks nor log segments found. Creating new log.
I20260925 07:27:58.936733 20522 tablet_bootstrap.cc:654] T faedc57931424efc8451ff0e05e1661b P 7cce4a7fbbd34fbd86be2ab1c4671796: Neither blocks nor log segments found. Creating new log.
I20260925 07:27:58.937240 20521 log.cc:824] T faedc57931424efc8451ff0e05e1661b P 308d38b0e2fd42d7afe17e8340b87f16: Log is configured to *not* fsync() on all Append() calls
I20260925 07:27:58.937291 20522 log.cc:824] T faedc57931424efc8451ff0e05e1661b P 7cce4a7fbbd34fbd86be2ab1c4671796: Log is configured to *not* fsync() on all Append() calls
I20260925 07:27:58.937944 20521 tablet_bootstrap.cc:492] T faedc57931424efc8451ff0e05e1661b P 308d38b0e2fd42d7afe17e8340b87f16: No bootstrap required, opened a new log
I20260925 07:27:58.937944 20522 tablet_bootstrap.cc:492] T faedc57931424efc8451ff0e05e1661b P 7cce4a7fbbd34fbd86be2ab1c4671796: No bootstrap required, opened a new log
I20260925 07:27:58.938009 20521 ts_tablet_manager.cc:1397] T faedc57931424efc8451ff0e05e1661b P 308d38b0e2fd42d7afe17e8340b87f16: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260925 07:27:58.938009 20522 ts_tablet_manager.cc:1397] T faedc57931424efc8451ff0e05e1661b P 7cce4a7fbbd34fbd86be2ab1c4671796: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 07:27:58.939659 20521 raft_consensus.cc:348] T faedc57931424efc8451ff0e05e1661b P 308d38b0e2fd42d7afe17e8340b87f16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308d38b0e2fd42d7afe17e8340b87f16" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 35777 } } peers { permanent_uuid: "7cce4a7fbbd34fbd86be2ab1c4671796" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 38307 } } peers { permanent_uuid: "de941b832d3d46fc82275294327a08ac" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 35035 } }
I20260925 07:27:58.939667 20522 raft_consensus.cc:348] T faedc57931424efc8451ff0e05e1661b P 7cce4a7fbbd34fbd86be2ab1c4671796 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308d38b0e2fd42d7afe17e8340b87f16" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 35777 } } peers { permanent_uuid: "7cce4a7fbbd34fbd86be2ab1c4671796" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 38307 } } peers { permanent_uuid: "de941b832d3d46fc82275294327a08ac" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 35035 } }
I20260925 07:27:58.939808 20521 raft_consensus.cc:374] T faedc57931424efc8451ff0e05e1661b P 308d38b0e2fd42d7afe17e8340b87f16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:27:58.939807 20522 raft_consensus.cc:374] T faedc57931424efc8451ff0e05e1661b P 7cce4a7fbbd34fbd86be2ab1c4671796 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:27:58.939857 20522 raft_consensus.cc:729] T faedc57931424efc8451ff0e05e1661b P 7cce4a7fbbd34fbd86be2ab1c4671796 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cce4a7fbbd34fbd86be2ab1c4671796, State: Initialized, Role: FOLLOWER
I20260925 07:27:58.939857 20521 raft_consensus.cc:729] T faedc57931424efc8451ff0e05e1661b P 308d38b0e2fd42d7afe17e8340b87f16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 308d38b0e2fd42d7afe17e8340b87f16, State: Initialized, Role: FOLLOWER
I20260925 07:27:58.939970 20522 consensus_queue.cc:260] T faedc57931424efc8451ff0e05e1661b P 7cce4a7fbbd34fbd86be2ab1c4671796 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308d38b0e2fd42d7afe17e8340b87f16" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 35777 } } peers { permanent_uuid: "7cce4a7fbbd34fbd86be2ab1c4671796" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 38307 } } peers { permanent_uuid: "de941b832d3d46fc82275294327a08ac" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 35035 } }
I20260925 07:27:58.939970 20521 consensus_queue.cc:260] T faedc57931424efc8451ff0e05e1661b P 308d38b0e2fd42d7afe17e8340b87f16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308d38b0e2fd42d7afe17e8340b87f16" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 35777 } } peers { permanent_uuid: "7cce4a7fbbd34fbd86be2ab1c4671796" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 38307 } } peers { permanent_uuid: "de941b832d3d46fc82275294327a08ac" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 35035 } }
I20260925 07:27:58.940225 20522 ts_tablet_manager.cc:1428] T faedc57931424efc8451ff0e05e1661b P 7cce4a7fbbd34fbd86be2ab1c4671796: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 07:27:58.940274 20511 heartbeater.cc:503] Master 127.5.158.126:43241 was elected leader, sending a full tablet report...
I20260925 07:27:58.940339 20379 heartbeater.cc:503] Master 127.5.158.126:43241 was elected leader, sending a full tablet report...
I20260925 07:27:58.940225 20521 ts_tablet_manager.cc:1428] T faedc57931424efc8451ff0e05e1661b P 308d38b0e2fd42d7afe17e8340b87f16: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
W20260925 07:27:59.040716 20380 tablet.cc:2369] T faedc57931424efc8451ff0e05e1661b P 7cce4a7fbbd34fbd86be2ab1c4671796: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 07:27:59.152012 20512 tablet.cc:2369] T faedc57931424efc8451ff0e05e1661b P 308d38b0e2fd42d7afe17e8340b87f16: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 07:27:59.178025 20242 tablet.cc:2369] T faedc57931424efc8451ff0e05e1661b P de941b832d3d46fc82275294327a08ac: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 07:27:59.194136 20526 raft_consensus.cc:482] T faedc57931424efc8451ff0e05e1661b P 308d38b0e2fd42d7afe17e8340b87f16 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 07:27:59.194229 20526 raft_consensus.cc:504] T faedc57931424efc8451ff0e05e1661b P 308d38b0e2fd42d7afe17e8340b87f16 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308d38b0e2fd42d7afe17e8340b87f16" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 35777 } } peers { permanent_uuid: "7cce4a7fbbd34fbd86be2ab1c4671796" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 38307 } } peers { permanent_uuid: "de941b832d3d46fc82275294327a08ac" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 35035 } }
I20260925 07:27:59.194619 20526 leader_election.cc:288] T faedc57931424efc8451ff0e05e1661b P 308d38b0e2fd42d7afe17e8340b87f16 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cce4a7fbbd34fbd86be2ab1c4671796 (127.5.158.66:38307), de941b832d3d46fc82275294327a08ac (127.5.158.65:35035)
I20260925 07:27:59.198467 20249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:59.194964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:34197 (local address 127.5.158.65:35035)
0925 07:27:59.195024 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:59.195026 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:59.195095 (+ 69us) server_negotiation.cc:407] Connection header received
0925 07:27:59.195125 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:59.195126 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:59.195148 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:59.195219 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:27:59.195434 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:59.196203 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:59.197616 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:59.198169 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:59.198228 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:59.198283 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:59.198369 (+ 86us) server_negotiation.cc:299] Negotiation successful
0925 07:27:59.198409 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 07:27:59.198467 20519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:59.194917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:53073 (local address 127.5.158.66:38307)
0925 07:27:59.194960 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:59.194962 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:59.194972 (+ 10us) server_negotiation.cc:407] Connection header received
0925 07:27:59.195065 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:59.195067 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:59.195078 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:59.195142 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:27:59.195434 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:59.196198 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:59.197616 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:59.198120 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:59.198198 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:59.198208 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:59.198369 (+ 161us) server_negotiation.cc:299] Negotiation successful
0925 07:27:59.198405 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 07:27:59.198518 20513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:59.194840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.66:38307 (local address 127.5.158.67:53073)
0925 07:27:59.194908 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:59.194920 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 07:27:59.194988 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:27:59.195163 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:27:59.195164 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:27:59.195166 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:27:59.195286 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:59.195292 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:59.196294 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:59.196295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:59.197533 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:59.197537 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:59.198215 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:59.198217 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:59.198295 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:59.198297 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 07:27:59.198349 (+ 52us) client_negotiation.cc:240] Negotiation successful
0925 07:27:59.198398 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":18}
I20260925 07:27:59.198523 20527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:59.194840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:35035 (local address 127.5.158.67:34197)
0925 07:27:59.195039 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:59.195047 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 07:27:59.195077 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:27:59.195227 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:27:59.195227 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:27:59.195229 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:27:59.195308 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:59.195309 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:59.196294 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:59.196295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:59.197533 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:27:59.197537 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:27:59.198233 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:27:59.198233 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:27:59.198292 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:59.198296 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 07:27:59.198349 (+ 53us) client_negotiation.cc:240] Negotiation successful
0925 07:27:59.198398 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"mutex_wait_us":147,"thread_start_us":103,"threads_started":1}
I20260925 07:27:59.198897 20195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "faedc57931424efc8451ff0e05e1661b" candidate_uuid: "308d38b0e2fd42d7afe17e8340b87f16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de941b832d3d46fc82275294327a08ac" is_pre_election: true
I20260925 07:27:59.198917 20333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "faedc57931424efc8451ff0e05e1661b" candidate_uuid: "308d38b0e2fd42d7afe17e8340b87f16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cce4a7fbbd34fbd86be2ab1c4671796" is_pre_election: true
I20260925 07:27:59.199007 20195 raft_consensus.cc:2454] T faedc57931424efc8451ff0e05e1661b P de941b832d3d46fc82275294327a08ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 308d38b0e2fd42d7afe17e8340b87f16 in term 0.
I20260925 07:27:59.199057 20333 raft_consensus.cc:2454] T faedc57931424efc8451ff0e05e1661b P 7cce4a7fbbd34fbd86be2ab1c4671796 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 308d38b0e2fd42d7afe17e8340b87f16 in term 0.
I20260925 07:27:59.199275 20399 leader_election.cc:302] T faedc57931424efc8451ff0e05e1661b P 308d38b0e2fd42d7afe17e8340b87f16 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 308d38b0e2fd42d7afe17e8340b87f16, 7cce4a7fbbd34fbd86be2ab1c4671796; no voters:
I20260925 07:27:59.199399 20526 raft_consensus.cc:2781] T faedc57931424efc8451ff0e05e1661b P 308d38b0e2fd42d7afe17e8340b87f16 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 07:27:59.199448 20526 raft_consensus.cc:482] T faedc57931424efc8451ff0e05e1661b P 308d38b0e2fd42d7afe17e8340b87f16 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 07:27:59.199479 20526 raft_consensus.cc:3037] T faedc57931424efc8451ff0e05e1661b P 308d38b0e2fd42d7afe17e8340b87f16 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:27:59.200120 20526 raft_consensus.cc:504] T faedc57931424efc8451ff0e05e1661b P 308d38b0e2fd42d7afe17e8340b87f16 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308d38b0e2fd42d7afe17e8340b87f16" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 35777 } } peers { permanent_uuid: "7cce4a7fbbd34fbd86be2ab1c4671796" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 38307 } } peers { permanent_uuid: "de941b832d3d46fc82275294327a08ac" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 35035 } }
I20260925 07:27:59.200248 20526 leader_election.cc:288] T faedc57931424efc8451ff0e05e1661b P 308d38b0e2fd42d7afe17e8340b87f16 [CANDIDATE]: Term 1 election: Requested vote from peers 7cce4a7fbbd34fbd86be2ab1c4671796 (127.5.158.66:38307), de941b832d3d46fc82275294327a08ac (127.5.158.65:35035)
I20260925 07:27:59.200356 20333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "faedc57931424efc8451ff0e05e1661b" candidate_uuid: "308d38b0e2fd42d7afe17e8340b87f16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cce4a7fbbd34fbd86be2ab1c4671796"
I20260925 07:27:59.200376 20195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "faedc57931424efc8451ff0e05e1661b" candidate_uuid: "308d38b0e2fd42d7afe17e8340b87f16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de941b832d3d46fc82275294327a08ac"
I20260925 07:27:59.200464 20333 raft_consensus.cc:3037] T faedc57931424efc8451ff0e05e1661b P 7cce4a7fbbd34fbd86be2ab1c4671796 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:27:59.200459 20195 raft_consensus.cc:3037] T faedc57931424efc8451ff0e05e1661b P de941b832d3d46fc82275294327a08ac [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:27:59.201239 20195 raft_consensus.cc:2454] T faedc57931424efc8451ff0e05e1661b P de941b832d3d46fc82275294327a08ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 308d38b0e2fd42d7afe17e8340b87f16 in term 1.
I20260925 07:27:59.201267 20333 raft_consensus.cc:2454] T faedc57931424efc8451ff0e05e1661b P 7cce4a7fbbd34fbd86be2ab1c4671796 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 308d38b0e2fd42d7afe17e8340b87f16 in term 1.
I20260925 07:27:59.201419 20399 leader_election.cc:302] T faedc57931424efc8451ff0e05e1661b P 308d38b0e2fd42d7afe17e8340b87f16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 308d38b0e2fd42d7afe17e8340b87f16, 7cce4a7fbbd34fbd86be2ab1c4671796; no voters:
I20260925 07:27:59.201568 20526 raft_consensus.cc:2781] T faedc57931424efc8451ff0e05e1661b P 308d38b0e2fd42d7afe17e8340b87f16 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:27:59.201710 20526 raft_consensus.cc:686] T faedc57931424efc8451ff0e05e1661b P 308d38b0e2fd42d7afe17e8340b87f16 [term 1 LEADER]: Becoming Leader. State: Replica: 308d38b0e2fd42d7afe17e8340b87f16, State: Running, Role: LEADER
I20260925 07:27:59.201799 20526 consensus_queue.cc:237] T faedc57931424efc8451ff0e05e1661b P 308d38b0e2fd42d7afe17e8340b87f16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308d38b0e2fd42d7afe17e8340b87f16" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 35777 } } peers { permanent_uuid: "7cce4a7fbbd34fbd86be2ab1c4671796" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 38307 } } peers { permanent_uuid: "de941b832d3d46fc82275294327a08ac" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 35035 } }
I20260925 07:27:59.202607 20053 catalog_manager.cc:5495] T faedc57931424efc8451ff0e05e1661b P 308d38b0e2fd42d7afe17e8340b87f16 reported cstate change: term changed from 0 to 1, leader changed from <none> to 308d38b0e2fd42d7afe17e8340b87f16 (127.5.158.67). New cstate: current_term: 1 leader_uuid: "308d38b0e2fd42d7afe17e8340b87f16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308d38b0e2fd42d7afe17e8340b87f16" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 35777 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cce4a7fbbd34fbd86be2ab1c4671796" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 38307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de941b832d3d46fc82275294327a08ac" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 35035 } health_report { overall_health: UNKNOWN } } }
I20260925 07:27:59.213110 20518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:59.210798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46714 (local address 127.5.158.67:35777)
0925 07:27:59.210868 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0925 07:27:59.210870 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:27:59.210878 (+ 8us) server_negotiation.cc:407] Connection header received
0925 07:27:59.210901 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:27:59.210902 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:27:59.210915 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:27:59.210990 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 07:27:59.211162 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:59.211760 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:59.212472 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:27:59.212644 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:27:59.212691 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:27:59.212818 (+ 127us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 07:27:59.212887 (+ 69us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 07:27:59.212936 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0925 07:27:59.212964 (+ 28us) server_negotiation.cc:299] Negotiation successful
0925 07:27:59.212977 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 07:27:59.214527 20333 raft_consensus.cc:1261] T faedc57931424efc8451ff0e05e1661b P 7cce4a7fbbd34fbd86be2ab1c4671796 [term 1 FOLLOWER]: Refusing update from remote peer 308d38b0e2fd42d7afe17e8340b87f16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:27:59.214533 20195 raft_consensus.cc:1261] T faedc57931424efc8451ff0e05e1661b P de941b832d3d46fc82275294327a08ac [term 1 FOLLOWER]: Refusing update from remote peer 308d38b0e2fd42d7afe17e8340b87f16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:27:59.214862 20528 consensus_queue.cc:1035] T faedc57931424efc8451ff0e05e1661b P 308d38b0e2fd42d7afe17e8340b87f16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cce4a7fbbd34fbd86be2ab1c4671796" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 38307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:27:59.215014 20526 consensus_queue.cc:1035] T faedc57931424efc8451ff0e05e1661b P 308d38b0e2fd42d7afe17e8340b87f16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de941b832d3d46fc82275294327a08ac" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 35035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:27:59.217031 20532 mvcc.cc:205] Tried to move back new op lower bound from 7333155959660670976 to 7333155959611219968. Current Snapshot: MvccSnapshot[applied={T|T < 7333155959660670976}]
I20260925 07:27:59.218844 20053 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:36158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 07:27:59.218923 20053 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:36158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 07:27:59.219533 20053 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9: Sending DeleteTablet for 3 replicas of tablet faedc57931424efc8451ff0e05e1661b
I20260925 07:27:59.219805 20300 tablet_service.cc:1514] Processing DeleteTablet for tablet faedc57931424efc8451ff0e05e1661b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:27:59 UTC) from {username='kudu', principal='kudu/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:54602
I20260925 07:27:59.219851 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 20091
I20260925 07:27:59.219858 20175 tablet_service.cc:1514] Processing DeleteTablet for tablet faedc57931424efc8451ff0e05e1661b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:27:59 UTC) from {username='kudu', principal='kudu/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:60858
I20260925 07:27:59.221268 20445 tablet_service.cc:1514] Processing DeleteTablet for tablet faedc57931424efc8451ff0e05e1661b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:27:59 UTC) from {username='kudu', principal='kudu/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:46698
W20260925 07:27:59.224783 20039 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.158.65:35035 (error 108)
I20260925 07:27:59.225018 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 20247
I20260925 07:27:59.225028 20541 tablet_replica.cc:330] T faedc57931424efc8451ff0e05e1661b P 7cce4a7fbbd34fbd86be2ab1c4671796: stopping tablet replica
I20260925 07:27:59.225138 20541 raft_consensus.cc:2229] T faedc57931424efc8451ff0e05e1661b P 7cce4a7fbbd34fbd86be2ab1c4671796 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 07:27:59.225237 20541 raft_consensus.cc:2258] T faedc57931424efc8451ff0e05e1661b P 7cce4a7fbbd34fbd86be2ab1c4671796 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:27:59.225785 20517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:59.225162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:35035 (local address 127.0.0.1:60872)
0925 07:27:59.225710 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:59.225757 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.65:35035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":530}
W20260925 07:27:59.225883 20039 catalog_manager.cc:4584] TS de941b832d3d46fc82275294327a08ac (127.5.158.65:35035): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet faedc57931424efc8451ff0e05e1661b: Network error: Client connection negotiation failed: client connection to 127.5.158.65:35035: connect: Connection refused (error 111)
I20260925 07:27:59.227180 20542 tablet_replica.cc:330] T faedc57931424efc8451ff0e05e1661b P 308d38b0e2fd42d7afe17e8340b87f16: stopping tablet replica
I20260925 07:27:59.227264 20542 raft_consensus.cc:2229] T faedc57931424efc8451ff0e05e1661b P 308d38b0e2fd42d7afe17e8340b87f16 [term 1 LEADER]: Raft consensus shutting down.
I20260925 07:27:59.227341 20542 raft_consensus.cc:2258] T faedc57931424efc8451ff0e05e1661b P 308d38b0e2fd42d7afe17e8340b87f16 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:27:59.227670 20542 ts_tablet_manager.cc:1905] T faedc57931424efc8451ff0e05e1661b P 308d38b0e2fd42d7afe17e8340b87f16: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:27:59.228966 20542 ts_tablet_manager.cc:1918] T faedc57931424efc8451ff0e05e1661b P 308d38b0e2fd42d7afe17e8340b87f16: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 07:27:59.229074 20542 log.cc:1196] T faedc57931424efc8451ff0e05e1661b P 308d38b0e2fd42d7afe17e8340b87f16: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-2/wal/wals/faedc57931424efc8451ff0e05e1661b
I20260925 07:27:59.229349 20542 ts_tablet_manager.cc:1939] T faedc57931424efc8451ff0e05e1661b P 308d38b0e2fd42d7afe17e8340b87f16: Deleting consensus metadata
I20260925 07:27:59.229775 20039 catalog_manager.cc:4841] TS 308d38b0e2fd42d7afe17e8340b87f16 (127.5.158.67:35777): tablet faedc57931424efc8451ff0e05e1661b (table test-table [id=b451171f6f4943beae4a03fd43fb1223]) successfully deleted
I20260925 07:27:59.230911 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 20382
I20260925 07:27:59.230924 20517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:59.230812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.66:38307 (local address 127.0.0.1:54618)
0925 07:27:59.230874 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:59.230898 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.66:38307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260925 07:27:59.236123 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 20020
I20260925 07:27:59.240732 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:43241
--webserver_interface=127.5.158.126
--webserver_port=43063
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:42351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.158.126:43241
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:27:59.312750 20544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:27:59.312906 20544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:27:59.312924 20544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:27:59.313952 20544 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:27:59.313992 20544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:27:59.314006 20544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:27:59.314019 20544 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:27:59.314031 20544 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:27:59.315326 20544 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:42351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.158.126:43241
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:43241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.158.126
--webserver_port=43063
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20544
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:27:59.315469 20544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:27:59.315596 20544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:27:59.319665 20549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:27:59.319779 20544 server_base.cc:1034] running on GCE node
W20260925 07:27:59.319665 20552 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:27:59.319854 20550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:27:59.320081 20544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:27:59.320324 20544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:27:59.321475 20544 hybrid_clock.cc:648] HybridClock initialized: now 1790321279321476 us; error 39 us; skew 500 ppm
Sep 25 07:27:59 dist-test-slave-1drq krb5kdc[19477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321279, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:27:59.323848 20544 init.cc:377] Logged in from keytab as kudu/127.5.158.126@KRBTEST.COM (short username kudu)
I20260925 07:27:59.324301 20544 webserver.cc:460] Webserver started at http://127.5.158.126:43063/ using document root <none> and password file <none>
I20260925 07:27:59.324509 20544 fs_manager.cc:359] Metadata directory not provided
I20260925 07:27:59.324577 20544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:27:59.325922 20544 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 07:27:59.326538 20559 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:59.326725 20544 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:27:59.326828 20544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "e8e5d0766eb541f5b6c08f1844b6d6e9"
format_stamp: "Formatted at 2026-09-25 07:27:57 on dist-test-slave-1drq"
server_key: "489dfa6637e0085f04459f57da95f3a2"
server_key_iv: "05622286d32ccad4a4b13f49ad505c53"
server_key_version: "encryptionkey@0"
I20260925 07:27:59.327113 20544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:27:59.340101 20544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:27:59.341946 20544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:27:59.342123 20544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:27:59.346109 20544 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.126:43241
I20260925 07:27:59.346159 20611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.126:43241 every 8 connection(s)
I20260925 07:27:59.346524 20544 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:27:59.347940 20612 sys_catalog.cc:263] Verifying existing consensus state
I20260925 07:27:59.348526 20612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9: Bootstrap starting.
I20260925 07:27:59.355191 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 20544
I20260925 07:27:59.355512 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.65:35035
--local_ip_for_outbound_sockets=127.5.158.65
--tserver_master_addrs=127.5.158.126:43241
--webserver_port=35217
--webserver_interface=127.5.158.65
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab
--principal=kudu/127.5.158.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:42351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 07:27:59.356594 20612 log.cc:824] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9: Log is configured to *not* fsync() on all Append() calls
I20260925 07:27:59.358564 20612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 07:27:59.358875 20612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9: Bootstrap complete.
I20260925 07:27:59.361073 20612 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8e5d0766eb541f5b6c08f1844b6d6e9" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 43241 } }
I20260925 07:27:59.361330 20612 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8e5d0766eb541f5b6c08f1844b6d6e9, State: Initialized, Role: FOLLOWER
I20260925 07:27:59.361495 20612 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8e5d0766eb541f5b6c08f1844b6d6e9" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 43241 } }
I20260925 07:27:59.361581 20612 raft_consensus.cc:388] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 07:27:59.361636 20612 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 07:27:59.361686 20612 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9 [term 2 FOLLOWER]: Advancing to term 3
I20260925 07:27:59.362618 20612 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8e5d0766eb541f5b6c08f1844b6d6e9" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 43241 } }
I20260925 07:27:59.362736 20612 leader_election.cc:302] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e8e5d0766eb541f5b6c08f1844b6d6e9; no voters:
I20260925 07:27:59.362924 20612 leader_election.cc:288] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260925 07:27:59.362978 20616 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9 [term 3 FOLLOWER]: Leader election won for term 3
I20260925 07:27:59.363086 20616 raft_consensus.cc:686] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9 [term 3 LEADER]: Becoming Leader. State: Replica: e8e5d0766eb541f5b6c08f1844b6d6e9, State: Running, Role: LEADER
I20260925 07:27:59.363185 20612 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:27:59.363173 20616 consensus_queue.cc:237] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8e5d0766eb541f5b6c08f1844b6d6e9" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 43241 } }
I20260925 07:27:59.363426 20616 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8e5d0766eb541f5b6c08f1844b6d6e9. Latest consensus state: current_term: 3 leader_uuid: "e8e5d0766eb541f5b6c08f1844b6d6e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8e5d0766eb541f5b6c08f1844b6d6e9" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 43241 } } }
I20260925 07:27:59.363487 20616 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9 [sys.catalog]: This master's current role is: LEADER
I20260925 07:27:59.365191 20618 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "e8e5d0766eb541f5b6c08f1844b6d6e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8e5d0766eb541f5b6c08f1844b6d6e9" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 43241 } } }
I20260925 07:27:59.365269 20618 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8e5d0766eb541f5b6c08f1844b6d6e9 [sys.catalog]: This master's current role is: LEADER
I20260925 07:27:59.365623 20620 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 07:27:59.366315 20620 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 07:27:59.366425 20620 catalog_manager.cc:1259] Loaded cluster ID: 6be6538bdebe4e52a9afcf87a34de8dd
I20260925 07:27:59.366458 20620 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260925 07:27:59.366650 20620 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f88f5bcc9ad google::LogMessage::Fail() at ??:0
@ 0x7f88f5bcf88a google::LogMessage::SendToLog() at ??:0
@ 0x7f88f5bcc4de google::LogMessage::Flush() at ??:0
@ 0x7f88f5bcda19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f88fdaed0f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f88fdaf68b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f88f7a9a0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f88f7a8f954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f88f88f36db start_thread at ??:0
@ 0x7f88f46ca61f clone at ??:0
W20260925 07:27:59.440382 20614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:27:59.440568 20614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:27:59.440601 20614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:27:59.441876 20614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:27:59.441939 20614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:27:59.441998 20614 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.65
I20260925 07:27:59.443377 20614 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:42351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab
--principal=kudu/127.5.158.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.65:35035
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.158.65
--webserver_port=35217
--tserver_master_addrs=127.5.158.126:43241
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20614
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.65
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:27:59.443617 20614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:27:59.443851 20614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:27:59.448273 20651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:27:59.448284 20648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:27:59.448328 20649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:27:59.448797 20614 server_base.cc:1034] running on GCE node
I20260925 07:27:59.448971 20614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:27:59.449239 20614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:27:59.450387 20614 hybrid_clock.cc:648] HybridClock initialized: now 1790321279450374 us; error 31 us; skew 500 ppm
Sep 25 07:27:59 dist-test-slave-1drq krb5kdc[19477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321279, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:27:59.452721 20614 init.cc:377] Logged in from keytab as kudu/127.5.158.65@KRBTEST.COM (short username kudu)
I20260925 07:27:59.453191 20614 webserver.cc:460] Webserver started at http://127.5.158.65:35217/ using document root <none> and password file <none>
I20260925 07:27:59.453408 20614 fs_manager.cc:359] Metadata directory not provided
I20260925 07:27:59.453474 20614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:27:59.454730 20614 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 07:27:59.455444 20658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:59.455615 20614 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 07:27:59.455704 20614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-0/wal
uuid: "de941b832d3d46fc82275294327a08ac"
format_stamp: "Formatted at 2026-09-25 07:27:57 on dist-test-slave-1drq"
server_key: "cf9e5c9c55e1d3ec00bba553e70b789d"
server_key_iv: "0eb4622a47088c9d8b19c21bf82e4729"
server_key_version: "encryptionkey@0"
I20260925 07:27:59.456002 20614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:27:59.498162 20614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:27:59.500327 20614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:27:59.500502 20614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:27:59.500738 20614 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:27:59.501349 20665 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260925 07:27:59.503186 20614 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260925 07:27:59.503253 20614 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260925 07:27:59.503296 20614 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260925 07:27:59.503958 20614 ts_tablet_manager.cc:610] Registered 1 tablets
I20260925 07:27:59.504056 20614 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260925 07:27:59.504118 20665 tablet_bootstrap.cc:492] T faedc57931424efc8451ff0e05e1661b P de941b832d3d46fc82275294327a08ac: Bootstrap starting.
I20260925 07:27:59.511287 20614 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.65:35035
I20260925 07:27:59.511410 20772 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.65:35035 every 8 connection(s)
I20260925 07:27:59.511687 20614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
I20260925 07:27:59.514320 20665 log.cc:824] T faedc57931424efc8451ff0e05e1661b P de941b832d3d46fc82275294327a08ac: Log is configured to *not* fsync() on all Append() calls
I20260925 07:27:59.516422 20665 tablet_bootstrap.cc:492] T faedc57931424efc8451ff0e05e1661b P de941b832d3d46fc82275294327a08ac: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 07:27:59.516726 20665 tablet_bootstrap.cc:492] T faedc57931424efc8451ff0e05e1661b P de941b832d3d46fc82275294327a08ac: Bootstrap complete.
I20260925 07:27:59.517611 20665 ts_tablet_manager.cc:1397] T faedc57931424efc8451ff0e05e1661b P de941b832d3d46fc82275294327a08ac: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.013s
I20260925 07:27:59.518585 20665 raft_consensus.cc:348] T faedc57931424efc8451ff0e05e1661b P de941b832d3d46fc82275294327a08ac [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308d38b0e2fd42d7afe17e8340b87f16" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 35777 } } peers { permanent_uuid: "7cce4a7fbbd34fbd86be2ab1c4671796" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 38307 } } peers { permanent_uuid: "de941b832d3d46fc82275294327a08ac" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 35035 } }
I20260925 07:27:59.518822 20665 raft_consensus.cc:729] T faedc57931424efc8451ff0e05e1661b P de941b832d3d46fc82275294327a08ac [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: de941b832d3d46fc82275294327a08ac, State: Initialized, Role: FOLLOWER
I20260925 07:27:59.518976 20665 consensus_queue.cc:260] T faedc57931424efc8451ff0e05e1661b P de941b832d3d46fc82275294327a08ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308d38b0e2fd42d7afe17e8340b87f16" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 35777 } } peers { permanent_uuid: "7cce4a7fbbd34fbd86be2ab1c4671796" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 38307 } } peers { permanent_uuid: "de941b832d3d46fc82275294327a08ac" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 35035 } }
I20260925 07:27:59.519222 20665 ts_tablet_manager.cc:1428] T faedc57931424efc8451ff0e05e1661b P de941b832d3d46fc82275294327a08ac: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260925 07:27:59.520422 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 20614
I20260925 07:27:59.520735 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.66:38307
--local_ip_for_outbound_sockets=127.5.158.66
--tserver_master_addrs=127.5.158.126:43241
--webserver_port=42683
--webserver_interface=127.5.158.66
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab
--principal=kudu/127.5.158.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:42351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:27:59.558159 20775 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:27:59.512489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.65:39837)
0925 07:27:59.512695 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:59.512759 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0925 07:27:59.513485 (+ 726us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:27:59.557977 (+ 44492us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":106,"threads_started":1}
W20260925 07:27:59.558375 20773 heartbeater.cc:644] Failed to heartbeat to 127.5.158.126:43241 (0 consecutive failures): Network error: Failed to ping master at 127.5.158.126:43241: Client connection negotiation failed: client connection to 127.5.158.126:43241: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 07:27:59.558738 20775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:59.558638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.65:54759)
0925 07:27:59.558686 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:59.558721 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260925 07:27:59.559048 20775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:59.558963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.65:59001)
0925 07:27:59.559007 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:59.559034 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260925 07:27:59.559139 20773 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 07:27:59.597157 20778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:27:59.597337 20778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:27:59.597373 20778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:27:59.598380 20778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:27:59.598443 20778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:27:59.598484 20778 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.66
I20260925 07:27:59.599699 20778 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:42351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab
--principal=kudu/127.5.158.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.66:38307
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.158.66
--webserver_port=42683
--tserver_master_addrs=127.5.158.126:43241
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20778
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.66
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:27:59.599917 20778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:27:59.600133 20778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:27:59.604693 20786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:27:59.604709 20778 server_base.cc:1034] running on GCE node
W20260925 07:27:59.604683 20784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:27:59.604683 20783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:27:59.605229 20778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:27:59.605458 20778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:27:59.606612 20778 hybrid_clock.cc:648] HybridClock initialized: now 1790321279606601 us; error 23 us; skew 500 ppm
Sep 25 07:27:59 dist-test-slave-1drq krb5kdc[19477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321279, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:27:59.608827 20778 init.cc:377] Logged in from keytab as kudu/127.5.158.66@KRBTEST.COM (short username kudu)
I20260925 07:27:59.609262 20778 webserver.cc:460] Webserver started at http://127.5.158.66:42683/ using document root <none> and password file <none>
I20260925 07:27:59.609414 20778 fs_manager.cc:359] Metadata directory not provided
I20260925 07:27:59.609448 20778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:27:59.610486 20778 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 07:27:59.611058 20793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:59.611258 20778 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:59.611334 20778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-1/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-1/wal
uuid: "7cce4a7fbbd34fbd86be2ab1c4671796"
format_stamp: "Formatted at 2026-09-25 07:27:57 on dist-test-slave-1drq"
server_key: "b1699fda57649e2a8ab6e67583325ffc"
server_key_iv: "335357c6c2f5b66a3fb7cc7d82052fcf"
server_key_version: "encryptionkey@0"
I20260925 07:27:59.611594 20778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:27:59.639770 20778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:27:59.641903 20778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:27:59.642056 20778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:27:59.642237 20778 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:27:59.642558 20800 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260925 07:27:59.643803 20778 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260925 07:27:59.643862 20778 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20260925 07:27:59.643901 20778 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260925 07:27:59.644454 20778 ts_tablet_manager.cc:610] Registered 1 tablets
I20260925 07:27:59.644508 20778 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260925 07:27:59.644567 20800 tablet_bootstrap.cc:492] T faedc57931424efc8451ff0e05e1661b P 7cce4a7fbbd34fbd86be2ab1c4671796: Bootstrap starting.
I20260925 07:27:59.651124 20778 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.66:38307
I20260925 07:27:59.651178 20907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.66:38307 every 8 connection(s)
I20260925 07:27:59.651515 20778 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
I20260925 07:27:59.652412 20910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:59.652092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.66:43263)
0925 07:27:59.652240 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:59.652301 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":77,"threads_started":1}
W20260925 07:27:59.652681 20908 heartbeater.cc:644] Failed to heartbeat to 127.5.158.126:43241 (0 consecutive failures): Network error: Failed to ping master at 127.5.158.126:43241: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
I20260925 07:27:59.652972 20910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:59.652895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.66:39085)
0925 07:27:59.652935 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:59.652957 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260925 07:27:59.653292 20910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:59.653221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.66:38461)
0925 07:27:59.653260 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:59.653279 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260925 07:27:59.653424 20908 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260925 07:27:59.653438 20800 log.cc:824] T faedc57931424efc8451ff0e05e1661b P 7cce4a7fbbd34fbd86be2ab1c4671796: Log is configured to *not* fsync() on all Append() calls
I20260925 07:27:59.655154 20800 tablet_bootstrap.cc:492] T faedc57931424efc8451ff0e05e1661b P 7cce4a7fbbd34fbd86be2ab1c4671796: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 07:27:59.655349 20800 tablet_bootstrap.cc:492] T faedc57931424efc8451ff0e05e1661b P 7cce4a7fbbd34fbd86be2ab1c4671796: Bootstrap complete.
I20260925 07:27:59.655822 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 20778
I20260925 07:27:59.656064 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.67:35777
--local_ip_for_outbound_sockets=127.5.158.67
--tserver_master_addrs=127.5.158.126:43241
--webserver_port=43081
--webserver_interface=127.5.158.67
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab
--principal=kudu/127.5.158.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:42351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 07:27:59.656126 20800 ts_tablet_manager.cc:1397] T faedc57931424efc8451ff0e05e1661b P 7cce4a7fbbd34fbd86be2ab1c4671796: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.003s
I20260925 07:27:59.657240 20800 raft_consensus.cc:348] T faedc57931424efc8451ff0e05e1661b P 7cce4a7fbbd34fbd86be2ab1c4671796 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308d38b0e2fd42d7afe17e8340b87f16" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 35777 } } peers { permanent_uuid: "7cce4a7fbbd34fbd86be2ab1c4671796" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 38307 } } peers { permanent_uuid: "de941b832d3d46fc82275294327a08ac" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 35035 } }
I20260925 07:27:59.657557 20800 raft_consensus.cc:729] T faedc57931424efc8451ff0e05e1661b P 7cce4a7fbbd34fbd86be2ab1c4671796 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cce4a7fbbd34fbd86be2ab1c4671796, State: Initialized, Role: FOLLOWER
I20260925 07:27:59.657667 20800 consensus_queue.cc:260] T faedc57931424efc8451ff0e05e1661b P 7cce4a7fbbd34fbd86be2ab1c4671796 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308d38b0e2fd42d7afe17e8340b87f16" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 35777 } } peers { permanent_uuid: "7cce4a7fbbd34fbd86be2ab1c4671796" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 38307 } } peers { permanent_uuid: "de941b832d3d46fc82275294327a08ac" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 35035 } }
I20260925 07:27:59.657883 20800 ts_tablet_manager.cc:1428] T faedc57931424efc8451ff0e05e1661b P 7cce4a7fbbd34fbd86be2ab1c4671796: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
I20260925 07:27:59.658102 20910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:59.658021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.66:49267)
0925 07:27:59.658057 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:59.658077 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260925 07:27:59.658339 20910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:59.658281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.66:48001)
0925 07:27:59.658304 (+ 23us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:59.658323 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260925 07:27:59.729081 20912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:27:59.729305 20912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:27:59.729326 20912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:27:59.730386 20912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:27:59.730424 20912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:27:59.730476 20912 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.67
I20260925 07:27:59.731814 20912 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:42351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab
--principal=kudu/127.5.158.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.67:35777
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.158.67
--webserver_port=43081
--tserver_master_addrs=127.5.158.126:43241
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20912
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.67
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:27:59.731957 20912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:27:59.732221 20912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:27:59.736871 20919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:27:59.736884 20921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:27:59.736900 20912 server_base.cc:1034] running on GCE node
W20260925 07:27:59.736886 20918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:27:59.737293 20912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:27:59.737469 20912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:27:59.738634 20912 hybrid_clock.cc:648] HybridClock initialized: now 1790321279738624 us; error 26 us; skew 500 ppm
Sep 25 07:27:59 dist-test-slave-1drq krb5kdc[19477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321279, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:27:59.740911 20912 init.cc:377] Logged in from keytab as kudu/127.5.158.67@KRBTEST.COM (short username kudu)
I20260925 07:27:59.741318 20912 webserver.cc:460] Webserver started at http://127.5.158.67:43081/ using document root <none> and password file <none>
I20260925 07:27:59.741482 20912 fs_manager.cc:359] Metadata directory not provided
I20260925 07:27:59.741520 20912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:27:59.742765 20912 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 07:27:59.743251 20928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:59.743409 20912 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:27:59.743479 20912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-2/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-2/wal
uuid: "308d38b0e2fd42d7afe17e8340b87f16"
format_stamp: "Formatted at 2026-09-25 07:27:58 on dist-test-slave-1drq"
server_key: "21cc47f0abc344fa8483396a74e5c19b"
server_key_iv: "71f98021aca893db8c042fb45cc56cda"
server_key_version: "encryptionkey@0"
I20260925 07:27:59.743728 20912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:27:59.755867 20912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:27:59.757862 20912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:27:59.757961 20912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:27:59.758105 20912 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:27:59.758322 20912 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:27:59.758347 20912 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:59.758365 20912 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:27:59.758376 20912 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:27:59.763949 20912 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.67:35777
I20260925 07:27:59.763999 21041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.67:35777 every 8 connection(s)
I20260925 07:27:59.764287 20912 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
I20260925 07:27:59.765228 21044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:59.764841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.67:57125)
0925 07:27:59.765038 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:59.765108 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":85,"threads_started":1}
W20260925 07:27:59.765573 21042 heartbeater.cc:644] Failed to heartbeat to 127.5.158.126:43241 (0 consecutive failures): Network error: Failed to ping master at 127.5.158.126:43241: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
I20260925 07:27:59.765862 21044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:59.765776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.67:57217)
0925 07:27:59.765823 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:59.765847 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260925 07:27:59.766120 21044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:27:59.766053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.67:60365)
0925 07:27:59.766089 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0925 07:27:59.766107 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260925 07:27:59.766247 21042 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260925 07:27:59.770648 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 20912
I20260925 07:28:00.559633 21046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:00.559434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.65:43103)
0925 07:28:00.559554 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:00.559583 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":69,"threads_started":1}
I20260925 07:28:00.658808 21047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:00.658633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.66:38877)
0925 07:28:00.658750 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:00.658778 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":54,"threads_started":1}
I20260925 07:28:00.726612 21048 raft_consensus.cc:482] T faedc57931424efc8451ff0e05e1661b P de941b832d3d46fc82275294327a08ac [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 07:28:00.726719 21048 raft_consensus.cc:504] T faedc57931424efc8451ff0e05e1661b P de941b832d3d46fc82275294327a08ac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308d38b0e2fd42d7afe17e8340b87f16" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 35777 } } peers { permanent_uuid: "7cce4a7fbbd34fbd86be2ab1c4671796" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 38307 } } peers { permanent_uuid: "de941b832d3d46fc82275294327a08ac" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 35035 } }
I20260925 07:28:00.727169 21048 leader_election.cc:288] T faedc57931424efc8451ff0e05e1661b P de941b832d3d46fc82275294327a08ac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 308d38b0e2fd42d7afe17e8340b87f16 (127.5.158.67:35777), 7cce4a7fbbd34fbd86be2ab1c4671796 (127.5.158.66:38307)
Sep 25 07:28:00 dist-test-slave-1drq krb5kdc[19477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321279, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.65@KRBTEST.COM for kudu/127.5.158.66@KRBTEST.COM
Sep 25 07:28:00 dist-test-slave-1drq krb5kdc[19477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321279, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.65@KRBTEST.COM for kudu/127.5.158.67@KRBTEST.COM
I20260925 07:28:00.735852 21046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:00.727308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.66:38307 (local address 127.5.158.65:38223)
0925 07:28:00.727359 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:00.727369 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 07:28:00.727419 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:28:00.727892 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:28:00.727895 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:28:00.728265 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:28:00.728483 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:28:00.728509 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:28:00.729547 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:28:00.729548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:28:00.730227 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:28:00.730229 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:28:00.730752 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:28:00.730765 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:28:00.730856 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:28:00.731338 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:28:00.731345 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:28:00.732543 (+ 1198us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:28:00.734484 (+ 1941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:28:00.734491 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:28:00.734493 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:28:00.734701 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:28:00.734970 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:28:00.734970 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:28:00.734971 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:28:00.735077 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:28:00.735423 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:28:00.735428 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:28:00.735535 (+ 107us) client_negotiation.cc:769] Sending connection context
0925 07:28:00.735622 (+ 87us) client_negotiation.cc:240] Negotiation successful
0925 07:28:00.735767 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":2}
I20260925 07:28:00.735875 21051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:00.727422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:38223 (local address 127.5.158.66:38307)
0925 07:28:00.727559 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0925 07:28:00.727568 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 07:28:00.727588 (+ 20us) server_negotiation.cc:407] Connection header received
0925 07:28:00.727686 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:28:00.727688 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:28:00.727726 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:28:00.727834 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:28:00.728606 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:28:00.729461 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:28:00.730315 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:28:00.730652 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:28:00.730782 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:28:00.732640 (+ 1858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:28:00.732655 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:28:00.732655 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:28:00.732663 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:28:00.734379 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:28:00.734792 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:28:00.734795 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:28:00.734795 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:28:00.734883 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:28:00.735151 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:28:00.735152 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:28:00.735152 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:28:00.735353 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:28:00.735427 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 07:28:00.735679 (+ 252us) server_negotiation.cc:299] Negotiation successful
0925 07:28:00.735784 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":46,"threads_started":1}
I20260925 07:28:00.735883 21050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:00.727400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:59483 (local address 127.5.158.67:35777)
0925 07:28:00.727544 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0925 07:28:00.727555 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 07:28:00.727577 (+ 22us) server_negotiation.cc:407] Connection header received
0925 07:28:00.728088 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:28:00.728090 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:28:00.728147 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:28:00.728271 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:28:00.728614 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:28:00.729473 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:28:00.730312 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:28:00.730646 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:28:00.730772 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:28:00.732965 (+ 2193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:28:00.732968 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:28:00.732968 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:28:00.732976 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:28:00.734561 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:28:00.734930 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:28:00.734933 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:28:00.734933 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:28:00.735027 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:28:00.735200 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:28:00.735200 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:28:00.735201 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:28:00.735385 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:28:00.735461 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 07:28:00.735674 (+ 213us) server_negotiation.cc:299] Negotiation successful
0925 07:28:00.735797 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":70,"threads_started":1}
I20260925 07:28:00.736001 21049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:00.727321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.67:35777 (local address 127.5.158.65:59483)
0925 07:28:00.727435 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:00.727444 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 07:28:00.727481 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:28:00.728260 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:28:00.728261 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:28:00.728400 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:28:00.728483 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:28:00.728509 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:28:00.729578 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:28:00.729578 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:28:00.730230 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:28:00.730232 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:28:00.730738 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:28:00.730739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:28:00.730856 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:28:00.731375 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:28:00.731379 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:28:00.732864 (+ 1485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:28:00.734673 (+ 1809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:28:00.734674 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:28:00.734675 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:28:00.734838 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:28:00.735087 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:28:00.735088 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:28:00.735088 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:28:00.735140 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:28:00.735461 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:28:00.735462 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:28:00.735539 (+ 77us) client_negotiation.cc:769] Sending connection context
0925 07:28:00.735618 (+ 79us) client_negotiation.cc:240] Negotiation successful
0925 07:28:00.735834 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"mutex_wait_us":162,"thread_start_us":58,"threads_started":1}
I20260925 07:28:00.736869 20862 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "faedc57931424efc8451ff0e05e1661b" candidate_uuid: "de941b832d3d46fc82275294327a08ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "7cce4a7fbbd34fbd86be2ab1c4671796" is_pre_election: true
I20260925 07:28:00.737008 20862 raft_consensus.cc:2454] T faedc57931424efc8451ff0e05e1661b P 7cce4a7fbbd34fbd86be2ab1c4671796 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de941b832d3d46fc82275294327a08ac in term 1.
W20260925 07:28:00.737159 20661 leader_election.cc:341] T faedc57931424efc8451ff0e05e1661b P de941b832d3d46fc82275294327a08ac [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 308d38b0e2fd42d7afe17e8340b87f16 (127.5.158.67:35777): Not found: Tablet not found: faedc57931424efc8451ff0e05e1661b
I20260925 07:28:00.737229 20660 leader_election.cc:302] T faedc57931424efc8451ff0e05e1661b P de941b832d3d46fc82275294327a08ac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cce4a7fbbd34fbd86be2ab1c4671796, de941b832d3d46fc82275294327a08ac; no voters: 308d38b0e2fd42d7afe17e8340b87f16
I20260925 07:28:00.737390 21048 raft_consensus.cc:2781] T faedc57931424efc8451ff0e05e1661b P de941b832d3d46fc82275294327a08ac [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260925 07:28:00.737439 21048 raft_consensus.cc:482] T faedc57931424efc8451ff0e05e1661b P de941b832d3d46fc82275294327a08ac [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 07:28:00.737452 21048 raft_consensus.cc:3037] T faedc57931424efc8451ff0e05e1661b P de941b832d3d46fc82275294327a08ac [term 1 FOLLOWER]: Advancing to term 2
I20260925 07:28:00.738386 21048 raft_consensus.cc:504] T faedc57931424efc8451ff0e05e1661b P de941b832d3d46fc82275294327a08ac [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308d38b0e2fd42d7afe17e8340b87f16" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 35777 } } peers { permanent_uuid: "7cce4a7fbbd34fbd86be2ab1c4671796" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 38307 } } peers { permanent_uuid: "de941b832d3d46fc82275294327a08ac" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 35035 } }
I20260925 07:28:00.738509 21048 leader_election.cc:288] T faedc57931424efc8451ff0e05e1661b P de941b832d3d46fc82275294327a08ac [CANDIDATE]: Term 2 election: Requested vote from peers 308d38b0e2fd42d7afe17e8340b87f16 (127.5.158.67:35777), 7cce4a7fbbd34fbd86be2ab1c4671796 (127.5.158.66:38307)
I20260925 07:28:00.738677 20862 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "faedc57931424efc8451ff0e05e1661b" candidate_uuid: "de941b832d3d46fc82275294327a08ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "7cce4a7fbbd34fbd86be2ab1c4671796"
W20260925 07:28:00.738729 20661 leader_election.cc:341] T faedc57931424efc8451ff0e05e1661b P de941b832d3d46fc82275294327a08ac [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 308d38b0e2fd42d7afe17e8340b87f16 (127.5.158.67:35777): Not found: Tablet not found: faedc57931424efc8451ff0e05e1661b
I20260925 07:28:00.738757 20862 raft_consensus.cc:3037] T faedc57931424efc8451ff0e05e1661b P 7cce4a7fbbd34fbd86be2ab1c4671796 [term 1 FOLLOWER]: Advancing to term 2
I20260925 07:28:00.739727 20862 raft_consensus.cc:2454] T faedc57931424efc8451ff0e05e1661b P 7cce4a7fbbd34fbd86be2ab1c4671796 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de941b832d3d46fc82275294327a08ac in term 2.
I20260925 07:28:00.739874 20660 leader_election.cc:302] T faedc57931424efc8451ff0e05e1661b P de941b832d3d46fc82275294327a08ac [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cce4a7fbbd34fbd86be2ab1c4671796, de941b832d3d46fc82275294327a08ac; no voters: 308d38b0e2fd42d7afe17e8340b87f16
I20260925 07:28:00.739959 21048 raft_consensus.cc:2781] T faedc57931424efc8451ff0e05e1661b P de941b832d3d46fc82275294327a08ac [term 2 FOLLOWER]: Leader election won for term 2
I20260925 07:28:00.740072 21048 raft_consensus.cc:686] T faedc57931424efc8451ff0e05e1661b P de941b832d3d46fc82275294327a08ac [term 2 LEADER]: Becoming Leader. State: Replica: de941b832d3d46fc82275294327a08ac, State: Running, Role: LEADER
I20260925 07:28:00.740147 21048 consensus_queue.cc:237] T faedc57931424efc8451ff0e05e1661b P de941b832d3d46fc82275294327a08ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308d38b0e2fd42d7afe17e8340b87f16" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 35777 } } peers { permanent_uuid: "7cce4a7fbbd34fbd86be2ab1c4671796" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 38307 } } peers { permanent_uuid: "de941b832d3d46fc82275294327a08ac" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 35035 } }
I20260925 07:28:00.740310 21049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:00.740242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.65:42317)
0925 07:28:00.740267 (+ 25us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:00.740287 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
I20260925 07:28:00.766603 21054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:00.766480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.67:40169)
0925 07:28:00.766548 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:00.766579 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58,"thread_start_us":34,"threads_started":1}
W20260925 07:28:01.222200 20661 consensus_peers.cc:487] T faedc57931424efc8451ff0e05e1661b P de941b832d3d46fc82275294327a08ac -> Peer 308d38b0e2fd42d7afe17e8340b87f16 (127.5.158.67:35777): Couldn't send request to peer 308d38b0e2fd42d7afe17e8340b87f16. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: faedc57931424efc8451ff0e05e1661b. This is attempt 1: this message will repeat every 5th retry.
I20260925 07:28:01.271250 20862 raft_consensus.cc:1261] T faedc57931424efc8451ff0e05e1661b P 7cce4a7fbbd34fbd86be2ab1c4671796 [term 2 FOLLOWER]: Refusing update from remote peer de941b832d3d46fc82275294327a08ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260925 07:28:01.271587 21048 consensus_queue.cc:1035] T faedc57931424efc8451ff0e05e1661b P de941b832d3d46fc82275294327a08ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cce4a7fbbd34fbd86be2ab1c4671796" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 38307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260925 07:28:01.272487 21057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:01.272263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.66:46683)
0925 07:28:01.272358 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:01.272412 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77,"thread_start_us":44,"threads_started":1}
I20260925 07:28:01.273305 21061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:01.273128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.65:46805)
0925 07:28:01.273216 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:01.273264 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77,"thread_start_us":37,"threads_started":1}
I20260925 07:28:01.273303 21057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:01.273209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.66:46869)
0925 07:28:01.273263 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:01.273285 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260925 07:28:01.627107 21062 ts_tablet_manager.cc:927] T faedc57931424efc8451ff0e05e1661b P 308d38b0e2fd42d7afe17e8340b87f16: Initiating tablet copy from peer de941b832d3d46fc82275294327a08ac (127.5.158.65:35035)
I20260925 07:28:01.627332 21062 tablet_copy_client.cc:316] T faedc57931424efc8451ff0e05e1661b P 308d38b0e2fd42d7afe17e8340b87f16: tablet copy: Beginning tablet copy session from remote peer at address 127.5.158.65:35035
Sep 25 07:28:01 dist-test-slave-1drq krb5kdc[19477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321279, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.67@KRBTEST.COM for kudu/127.5.158.65@KRBTEST.COM
I20260925 07:28:01.634696 21065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:01.627727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:40381 (local address 127.5.158.65:35035)
0925 07:28:01.627908 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0925 07:28:01.627913 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 07:28:01.627923 (+ 10us) server_negotiation.cc:407] Connection header received
0925 07:28:01.627954 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:28:01.627957 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:28:01.627969 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:28:01.628047 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:28:01.628509 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:28:01.629269 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:28:01.629966 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:28:01.630252 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:28:01.630326 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:28:01.632091 (+ 1765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:28:01.632099 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:28:01.632099 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:28:01.632106 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:28:01.633437 (+ 1331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:28:01.633847 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:28:01.633850 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:28:01.633851 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:28:01.633908 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:28:01.634118 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:28:01.634119 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:28:01.634119 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:28:01.634255 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:28:01.634343 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0925 07:28:01.634519 (+ 176us) server_negotiation.cc:299] Negotiation successful
0925 07:28:01.634604 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":74,"threads_started":1}
I20260925 07:28:01.634703 21064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:01.627604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:35035 (local address 127.5.158.67:40381)
0925 07:28:01.627753 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:01.627766 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 07:28:01.627842 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:28:01.628050 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:28:01.628055 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:28:01.628373 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:28:01.628427 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:28:01.628428 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:28:01.629345 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:28:01.629346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:28:01.629884 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:28:01.629886 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:28:01.630323 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:28:01.630324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:28:01.630404 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:28:01.630878 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:28:01.630883 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:28:01.632012 (+ 1129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:28:01.633543 (+ 1531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:28:01.633549 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:28:01.633551 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:28:01.633736 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:28:01.633975 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:28:01.633976 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:28:01.633976 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:28:01.634038 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:28:01.634332 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:28:01.634337 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:28:01.634388 (+ 51us) client_negotiation.cc:769] Sending connection context
0925 07:28:01.634486 (+ 98us) client_negotiation.cc:240] Negotiation successful
0925 07:28:01.634597 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":87,"threads_started":1}
I20260925 07:28:01.635212 20747 tablet_copy_service.cc:140] P de941b832d3d46fc82275294327a08ac: Received BeginTabletCopySession request for tablet faedc57931424efc8451ff0e05e1661b from peer 308d38b0e2fd42d7afe17e8340b87f16 ({username='kudu', principal='kudu/127.5.158.67@KRBTEST.COM'} at 127.5.158.67:40381)
I20260925 07:28:01.635267 20747 tablet_copy_service.cc:161] P de941b832d3d46fc82275294327a08ac: Beginning new tablet copy session on tablet faedc57931424efc8451ff0e05e1661b from peer 308d38b0e2fd42d7afe17e8340b87f16 at {username='kudu', principal='kudu/127.5.158.67@KRBTEST.COM'} at 127.5.158.67:40381: session id = 308d38b0e2fd42d7afe17e8340b87f16-faedc57931424efc8451ff0e05e1661b
I20260925 07:28:01.635728 20747 tablet_copy_source_session.cc:205] T faedc57931424efc8451ff0e05e1661b P de941b832d3d46fc82275294327a08ac: Tablet Copy: opened 0 blocks and 1 log segments
I20260925 07:28:01.636611 21062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet faedc57931424efc8451ff0e05e1661b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:28:01.638957 21062 tablet_copy_client.cc:791] T faedc57931424efc8451ff0e05e1661b P 308d38b0e2fd42d7afe17e8340b87f16: tablet copy: Starting download of 0 data blocks...
I20260925 07:28:01.639104 21062 tablet_copy_client.cc:655] T faedc57931424efc8451ff0e05e1661b P 308d38b0e2fd42d7afe17e8340b87f16: tablet copy: Starting download of 1 WAL segments...
I20260925 07:28:01.639824 21062 tablet_copy_client.cc:531] T faedc57931424efc8451ff0e05e1661b P 308d38b0e2fd42d7afe17e8340b87f16: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260925 07:28:01.640444 21062 tablet_bootstrap.cc:492] T faedc57931424efc8451ff0e05e1661b P 308d38b0e2fd42d7afe17e8340b87f16: Bootstrap starting.
I20260925 07:28:01.642163 21062 log.cc:824] T faedc57931424efc8451ff0e05e1661b P 308d38b0e2fd42d7afe17e8340b87f16: Log is configured to *not* fsync() on all Append() calls
I20260925 07:28:01.644532 21062 tablet_bootstrap.cc:492] T faedc57931424efc8451ff0e05e1661b P 308d38b0e2fd42d7afe17e8340b87f16: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 07:28:01.644814 21062 tablet_bootstrap.cc:492] T faedc57931424efc8451ff0e05e1661b P 308d38b0e2fd42d7afe17e8340b87f16: Bootstrap complete.
I20260925 07:28:01.644990 21062 ts_tablet_manager.cc:1397] T faedc57931424efc8451ff0e05e1661b P 308d38b0e2fd42d7afe17e8340b87f16: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260925 07:28:01.646107 21062 raft_consensus.cc:348] T faedc57931424efc8451ff0e05e1661b P 308d38b0e2fd42d7afe17e8340b87f16 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308d38b0e2fd42d7afe17e8340b87f16" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 35777 } } peers { permanent_uuid: "7cce4a7fbbd34fbd86be2ab1c4671796" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 38307 } } peers { permanent_uuid: "de941b832d3d46fc82275294327a08ac" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 35035 } }
I20260925 07:28:01.646255 21062 raft_consensus.cc:729] T faedc57931424efc8451ff0e05e1661b P 308d38b0e2fd42d7afe17e8340b87f16 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 308d38b0e2fd42d7afe17e8340b87f16, State: Initialized, Role: FOLLOWER
I20260925 07:28:01.646394 21062 consensus_queue.cc:260] T faedc57931424efc8451ff0e05e1661b P 308d38b0e2fd42d7afe17e8340b87f16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308d38b0e2fd42d7afe17e8340b87f16" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 35777 } } peers { permanent_uuid: "7cce4a7fbbd34fbd86be2ab1c4671796" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 38307 } } peers { permanent_uuid: "de941b832d3d46fc82275294327a08ac" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 35035 } }
I20260925 07:28:01.646620 21062 ts_tablet_manager.cc:1428] T faedc57931424efc8451ff0e05e1661b P 308d38b0e2fd42d7afe17e8340b87f16: Time spent starting tablet: real 0.002s user 0.001s sys 0.003s
I20260925 07:28:01.646869 20747 tablet_copy_service.cc:342] P de941b832d3d46fc82275294327a08ac: Request end of tablet copy session 308d38b0e2fd42d7afe17e8340b87f16-faedc57931424efc8451ff0e05e1661b received from {username='kudu', principal='kudu/127.5.158.67@KRBTEST.COM'} at 127.5.158.67:40381
I20260925 07:28:01.646886 21064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:01.646788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.67:60671)
0925 07:28:01.646827 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:01.646856 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260925 07:28:01.646934 20747 tablet_copy_service.cc:433] P de941b832d3d46fc82275294327a08ac: ending tablet copy session 308d38b0e2fd42d7afe17e8340b87f16-faedc57931424efc8451ff0e05e1661b on tablet faedc57931424efc8451ff0e05e1661b with peer 308d38b0e2fd42d7afe17e8340b87f16
I20260925 07:28:01.647181 21064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:01.647098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.67:56351)
0925 07:28:01.647128 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:01.647157 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260925 07:28:02.057899 21070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:02.057734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.65:38057)
0925 07:28:02.057834 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:02.057856 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":56,"threads_started":1}
I20260925 07:28:02.057898 20996 raft_consensus.cc:1203] T faedc57931424efc8451ff0e05e1661b P 308d38b0e2fd42d7afe17e8340b87f16 [term 2 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[2.3-2.3] Dedup: 2.3->[]
I20260925 07:28:02.058295 21064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:02.058203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.67:49673)
0925 07:28:02.058234 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:02.058264 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260925 07:28:02.273811 21071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:02.273608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.66:35889)
0925 07:28:02.273720 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:02.273751 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":66,"threads_started":1}
I20260925 07:28:03.058446 21072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:03.058260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.65:48913)
0925 07:28:03.058373 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:03.058405 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":74,"threads_started":1}
I20260925 07:28:03.058872 21073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:03.058638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.67:52699)
0925 07:28:03.058798 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:03.058837 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":43,"threads_started":1}
I20260925 07:28:03.274402 21074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:03.274209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.66:42727)
0925 07:28:03.274323 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:03.274356 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":53,"threads_started":1}
I20260925 07:28:04.059020 21075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:04.058848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.65:34817)
0925 07:28:04.058950 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:04.058980 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":49,"threads_started":1}
I20260925 07:28:04.059453 21076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:04.059277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.67:51323)
0925 07:28:04.059377 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:04.059409 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":58,"threads_started":1}
I20260925 07:28:04.275034 21077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:04.274805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.66:34595)
0925 07:28:04.274936 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:04.274980 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":73,"threads_started":1}
I20260925 07:28:05.059645 21078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:05.059465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.65:40831)
0925 07:28:05.059573 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:05.059603 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":58,"threads_started":1}
I20260925 07:28:05.060016 21079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:05.059842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.67:44189)
0925 07:28:05.059945 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:05.059979 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":49,"threads_started":1}
I20260925 07:28:05.060161 21078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:05.060044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.65:51721)
0925 07:28:05.060113 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:05.060133 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260925 07:28:05.275791 21080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:05.275560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.66:50473)
0925 07:28:05.275695 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:05.275726 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":64,"threads_started":1}
I20260925 07:28:06.060684 21081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:06.060482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.67:41625)
0925 07:28:06.060618 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:06.060643 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":90,"threads_started":1}
I20260925 07:28:06.060700 21082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:06.060536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.65:57037)
0925 07:28:06.060637 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:06.060663 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82,"thread_start_us":60,"threads_started":1}
I20260925 07:28:06.276444 21083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:06.276239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.66:33463)
0925 07:28:06.276369 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:06.276401 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":70,"threads_started":1}
I20260925 07:28:07.061251 21084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:07.061041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.65:49855)
0925 07:28:07.061171 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:07.061210 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":84,"threads_started":1}
I20260925 07:28:07.061283 21085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:07.061056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.67:40851)
0925 07:28:07.061213 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:07.061241 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":97,"threads_started":1}
I20260925 07:28:07.277156 21086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:07.276917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.66:41023)
0925 07:28:07.277062 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:07.277091 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":68,"threads_started":1}
I20260925 07:28:08.061887 21087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:08.061668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.65:39929)
0925 07:28:08.061809 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:08.061841 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":65,"threads_started":1}
I20260925 07:28:08.061892 21088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:08.061668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.67:53669)
0925 07:28:08.061830 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:08.061856 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":67,"threads_started":1}
I20260925 07:28:08.277786 21089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:08.277591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.66:42845)
0925 07:28:08.277710 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:08.277743 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":59,"threads_started":1}
I20260925 07:28:09.062503 21090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:09.062281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.67:57765)
0925 07:28:09.062409 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:09.062458 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":62,"threads_started":1}
I20260925 07:28:09.062510 21091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:09.062315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.65:36827)
0925 07:28:09.062421 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:09.062451 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":67,"threads_started":1}
I20260925 07:28:09.278380 21092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:09.278173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.66:48079)
0925 07:28:09.278299 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:09.278334 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":53,"threads_started":1}
I20260925 07:28:10.063066 21094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:10.062881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.65:34443)
0925 07:28:10.063006 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:10.063030 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":44,"threads_started":1}
I20260925 07:28:10.063066 21093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:10.062860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.67:57611)
0925 07:28:10.062993 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:10.063025 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":62,"threads_started":1}
I20260925 07:28:10.278988 21095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:10.278786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.66:33125)
0925 07:28:10.278904 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:10.278947 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":66,"threads_started":1}
I20260925 07:28:11.063663 21096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:11.063454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.65:34629)
0925 07:28:11.063572 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:11.063609 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":50,"threads_started":1}
I20260925 07:28:11.063704 21097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:11.063487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.67:58417)
0925 07:28:11.063635 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:11.063666 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":97,"threads_started":1}
I20260925 07:28:11.279592 21098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:11.279388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.66:56155)
0925 07:28:11.279502 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:11.279536 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":58,"threads_started":1}
I20260925 07:28:12.064599 21099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:12.064374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.67:54617)
0925 07:28:12.064496 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:12.064534 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":63,"threads_started":1}
I20260925 07:28:12.064608 21100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:12.064372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.65:47383)
0925 07:28:12.064529 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:12.064562 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":66,"threads_started":1}
I20260925 07:28:12.280187 21101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:12.279985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.66:35197)
0925 07:28:12.280114 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:12.280145 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":67,"threads_started":1}
I20260925 07:28:13.065405 21102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:13.065175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.67:39081)
0925 07:28:13.065328 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:13.065364 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":62,"threads_started":1}
I20260925 07:28:13.065412 21103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:13.065201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.65:44849)
0925 07:28:13.065349 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:13.065376 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":78,"threads_started":1}
I20260925 07:28:13.280856 21104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:13.280625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.66:55053)
0925 07:28:13.280785 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:13.280812 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":97,"threads_started":1}
I20260925 07:28:14.065970 21105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:14.065765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.67:38065)
0925 07:28:14.065879 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:14.065913 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":52,"threads_started":1}
I20260925 07:28:14.065980 21106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:14.065783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.65:33163)
0925 07:28:14.065898 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:14.065927 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":65,"threads_started":1}
I20260925 07:28:14.281488 21107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:14.281284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43241 (local address 127.5.158.66:56643)
0925 07:28:14.281404 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:14.281444 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":56,"threads_started":1}
I20260925 07:28:14.775753 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 20614
I20260925 07:28:14.780205 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 20778
I20260925 07:28:14.784178 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 20912
2026-09-25T07:28:14Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18322 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 07:28:14 dist-test-slave-1drq krb5kdc[21111](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 07:28:14 dist-test-slave-1drq krb5kdc[21111](info): set up 2 sockets
Sep 25 07:28:14 dist-test-slave-1drq krb5kdc[21111](info): commencing operation
krb5kdc: starting...
W20260925 07:28:15.744673 5753 mini_kdc.cc:121] Time spent starting KDC: real 0.941s user 0.001s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Sep 25 07:28:15 dist-test-slave-1drq krb5kdc[21111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321295, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T07:28:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T07:28:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
I20260925 07:28:15.843559 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:32843
--webserver_interface=127.5.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:44845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.158.126:32843
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:28:15.912760 21124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:28:15.912987 21124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:28:15.913028 21124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:28:15.914192 21124 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260925 07:28:15.914258 21124 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260925 07:28:15.914278 21124 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260925 07:28:15.914296 21124 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:28:15.914330 21124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:28:15.914355 21124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:28:15.914383 21124 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:28:15.914412 21124 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:28:15.915882 21124 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:44845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.5.158.126:32843
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:32843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21124
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:28:15.916162 21124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:28:15.916357 21124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 07:28:15.921185 21124 server_base.cc:1034] running on GCE node
W20260925 07:28:15.921201 21130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:28:15.921228 21132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:28:15.921360 21129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:28:15.921707 21124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:28:15.922003 21124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:28:15.923147 21124 hybrid_clock.cc:648] HybridClock initialized: now 1790321295923138 us; error 30 us; skew 500 ppm
Sep 25 07:28:15 dist-test-slave-1drq krb5kdc[21111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321295, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:28:15.925644 21124 init.cc:377] Logged in from keytab as kudu/127.5.158.126@KRBTEST.COM (short username kudu)
I20260925 07:28:15.926062 21124 webserver.cc:460] Webserver started at http://127.5.158.126:44765/ using document root <none> and password file <none>
I20260925 07:28:15.926271 21124 fs_manager.cc:359] Metadata directory not provided
I20260925 07:28:15.926330 21124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:28:15.926477 21124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:28:15.927452 21124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/master-0/data/instance:
uuid: "94c99bc2c8fd47adb7b95b181d9246de"
format_stamp: "Formatted at 2026-09-25 07:28:15 on dist-test-slave-1drq"
server_key: "60aff4a17a032ff097c0baad88203862"
server_key_iv: "1d61bdd82e828a1a192ee19ad2b3e3d9"
server_key_version: "encryptionkey@0"
I20260925 07:28:15.927798 21124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/master-0/wal/instance:
uuid: "94c99bc2c8fd47adb7b95b181d9246de"
format_stamp: "Formatted at 2026-09-25 07:28:15 on dist-test-slave-1drq"
server_key: "60aff4a17a032ff097c0baad88203862"
server_key_iv: "1d61bdd82e828a1a192ee19ad2b3e3d9"
server_key_version: "encryptionkey@0"
I20260925 07:28:15.928973 21124 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 07:28:15.929611 21139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:28:15.929822 21124 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 07:28:15.929908 21124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "94c99bc2c8fd47adb7b95b181d9246de"
format_stamp: "Formatted at 2026-09-25 07:28:15 on dist-test-slave-1drq"
server_key: "60aff4a17a032ff097c0baad88203862"
server_key_iv: "1d61bdd82e828a1a192ee19ad2b3e3d9"
server_key_version: "encryptionkey@0"
I20260925 07:28:15.929977 21124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:28:15.944444 21124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:28:15.946630 21124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:28:15.946792 21124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:28:15.950634 21124 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.126:32843
I20260925 07:28:15.950699 21191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.126:32843 every 8 connection(s)
I20260925 07:28:15.950981 21124 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:28:15.951792 21192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:28:15.953890 21192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de: Bootstrap starting.
I20260925 07:28:15.954447 21192 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de: Neither blocks nor log segments found. Creating new log.
I20260925 07:28:15.954726 21192 log.cc:824] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de: Log is configured to *not* fsync() on all Append() calls
I20260925 07:28:15.955345 21192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de: No bootstrap required, opened a new log
I20260925 07:28:15.956809 21192 raft_consensus.cc:348] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94c99bc2c8fd47adb7b95b181d9246de" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 32843 } }
I20260925 07:28:15.956928 21192 raft_consensus.cc:374] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:28:15.956995 21192 raft_consensus.cc:729] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94c99bc2c8fd47adb7b95b181d9246de, State: Initialized, Role: FOLLOWER
I20260925 07:28:15.957217 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 21124
I20260925 07:28:15.957185 21192 consensus_queue.cc:260] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94c99bc2c8fd47adb7b95b181d9246de" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 32843 } }
I20260925 07:28:15.957276 21192 raft_consensus.cc:388] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 07:28:15.957316 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/master-0/wal/instance
I20260925 07:28:15.957319 21192 raft_consensus.cc:482] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 07:28:15.957361 21192 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:28:15.957495 5753 external_mini_cluster.cc:1428] Setting key 4a85de8b502905dabdea9087a20a1248
I20260925 07:28:15.957952 21192 raft_consensus.cc:504] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94c99bc2c8fd47adb7b95b181d9246de" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 32843 } }
I20260925 07:28:15.958055 21192 leader_election.cc:302] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 94c99bc2c8fd47adb7b95b181d9246de; no voters:
I20260925 07:28:15.958199 21192 leader_election.cc:288] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 07:28:15.958374 21197 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:28:15.958411 21192 sys_catalog.cc:564] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:28:15.958566 21197 raft_consensus.cc:686] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de [term 1 LEADER]: Becoming Leader. State: Replica: 94c99bc2c8fd47adb7b95b181d9246de, State: Running, Role: LEADER
I20260925 07:28:15.958673 21197 consensus_queue.cc:237] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94c99bc2c8fd47adb7b95b181d9246de" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 32843 } }
I20260925 07:28:15.959517 21199 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de [sys.catalog]: SysCatalogTable state changed. Reason: New leader 94c99bc2c8fd47adb7b95b181d9246de. Latest consensus state: current_term: 1 leader_uuid: "94c99bc2c8fd47adb7b95b181d9246de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94c99bc2c8fd47adb7b95b181d9246de" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 32843 } } }
I20260925 07:28:15.959605 21199 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de [sys.catalog]: This master's current role is: LEADER
I20260925 07:28:15.959718 21198 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "94c99bc2c8fd47adb7b95b181d9246de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94c99bc2c8fd47adb7b95b181d9246de" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 32843 } } }
I20260925 07:28:15.959766 21198 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de [sys.catalog]: This master's current role is: LEADER
I20260925 07:28:15.959904 21210 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 07:28:15.960451 21210 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 07:28:15.962494 21210 catalog_manager.cc:1347] Generated new cluster ID: c111839a6f3744caa17ed42fd6e29909
I20260925 07:28:15.962539 21210 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 07:28:15 dist-test-slave-1drq krb5kdc[21111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321295, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:28:15.968894 21196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:15.957951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54812 (local address 127.5.158.126:32843)
0925 07:28:15.958140 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0925 07:28:15.958148 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 07:28:15.958175 (+ 27us) server_negotiation.cc:407] Connection header received
0925 07:28:15.958286 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:28:15.958293 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:28:15.958356 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:28:15.958511 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:28:15.959356 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:28:15.960773 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:28:15.961630 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:28:15.962208 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:28:15.962321 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:28:15.964865 (+ 2544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:28:15.964869 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:28:15.964869 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:28:15.964879 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:28:15.967163 (+ 2284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:28:15.967500 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:28:15.967502 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:28:15.967502 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:28:15.967595 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:28:15.967755 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:28:15.967756 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:28:15.967756 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:28:15.968025 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:28:15.968139 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0925 07:28:15.968547 (+ 408us) server_negotiation.cc:299] Negotiation successful
0925 07:28:15.968664 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":92,"threads_started":1}
I20260925 07:28:15.989264 21210 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 07:28:15.989802 21210 catalog_manager.cc:1504] Loading token signing keys...
I20260925 07:28:15.995879 21210 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de: Generated new TSK 0
I20260925 07:28:15.996052 21210 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.158.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab.
Entry for principal kudu/127.5.158.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab.
I20260925 07:28:16.024581 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.65:0
--local_ip_for_outbound_sockets=127.5.158.65
--webserver_interface=127.5.158.65
--webserver_port=0
--tserver_master_addrs=127.5.158.126:32843
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab
--principal=kudu/127.5.158.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:44845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:28:16.093843 21218 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:28:16.094020 21218 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:28:16.094041 21218 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:28:16.095150 21218 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:28:16.095191 21218 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:28:16.095240 21218 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.65
I20260925 07:28:16.096616 21218 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:44845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab
--principal=kudu/127.5.158.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.158.65
--webserver_port=0
--tserver_master_addrs=127.5.158.126:32843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21218
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.65
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:28:16.096827 21218 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:28:16.097013 21218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 07:28:16.101733 21218 server_base.cc:1034] running on GCE node
W20260925 07:28:16.101686 21226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:28:16.101682 21223 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:28:16.101688 21224 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:28:16.102026 21218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:28:16.102198 21218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:28:16.103330 21218 hybrid_clock.cc:648] HybridClock initialized: now 1790321296103324 us; error 27 us; skew 500 ppm
Sep 25 07:28:16 dist-test-slave-1drq krb5kdc[21111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321296, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:28:16.109351 21218 init.cc:377] Logged in from keytab as kudu/127.5.158.65@KRBTEST.COM (short username kudu)
I20260925 07:28:16.109758 21218 webserver.cc:460] Webserver started at http://127.5.158.65:39241/ using document root <none> and password file <none>
I20260925 07:28:16.109934 21218 fs_manager.cc:359] Metadata directory not provided
I20260925 07:28:16.109983 21218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:28:16.110062 21218 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:28:16.110970 21218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-0/data/instance:
uuid: "3d9933f641294109bd395addc5ab8c70"
format_stamp: "Formatted at 2026-09-25 07:28:16 on dist-test-slave-1drq"
server_key: "a35782025adaf52818d49771bb8a133c"
server_key_iv: "1fc684a67be1a898a1cc4aadc88a85e5"
server_key_version: "encryptionkey@0"
I20260925 07:28:16.111263 21218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-0/wal/instance:
uuid: "3d9933f641294109bd395addc5ab8c70"
format_stamp: "Formatted at 2026-09-25 07:28:16 on dist-test-slave-1drq"
server_key: "a35782025adaf52818d49771bb8a133c"
server_key_iv: "1fc684a67be1a898a1cc4aadc88a85e5"
server_key_version: "encryptionkey@0"
I20260925 07:28:16.112370 21218 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 07:28:16.112989 21233 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:28:16.113157 21218 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:28:16.113227 21218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-0/wal
uuid: "3d9933f641294109bd395addc5ab8c70"
format_stamp: "Formatted at 2026-09-25 07:28:16 on dist-test-slave-1drq"
server_key: "a35782025adaf52818d49771bb8a133c"
server_key_iv: "1fc684a67be1a898a1cc4aadc88a85e5"
server_key_version: "encryptionkey@0"
I20260925 07:28:16.113271 21218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:28:16.124212 21218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:28:16.125854 21218 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:28:16.125939 21218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:28:16.126067 21218 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:28:16.126279 21218 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:28:16.126322 21218 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:28:16.126341 21218 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:28:16.126353 21218 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:28:16.131852 21218 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.65:37045
I20260925 07:28:16.131913 21346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.65:37045 every 8 connection(s)
I20260925 07:28:16.132210 21218 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
Sep 25 07:28:16 dist-test-slave-1drq krb5kdc[21111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321296, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.65@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:28:16.138253 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 21218
I20260925 07:28:16.138367 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-0/wal/instance
I20260925 07:28:16.138562 5753 external_mini_cluster.cc:1428] Setting key 897da82870f0df0232febd5b91a03916
I20260925 07:28:16.141741 21196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:16.132868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:44875 (local address 127.5.158.126:32843)
0925 07:28:16.132928 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0925 07:28:16.132931 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 07:28:16.133118 (+ 187us) server_negotiation.cc:407] Connection header received
0925 07:28:16.133676 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:28:16.133678 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:28:16.133692 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:28:16.133768 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:28:16.134679 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:28:16.135201 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:28:16.136202 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:28:16.136440 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:28:16.136523 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:28:16.138687 (+ 2164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:28:16.138689 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:28:16.138689 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:28:16.138706 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:28:16.140213 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:28:16.140746 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:28:16.140748 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:28:16.140748 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:28:16.140801 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:28:16.141119 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:28:16.141120 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:28:16.141121 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:28:16.141247 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:28:16.141294 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 07:28:16.141623 (+ 329us) server_negotiation.cc:299] Negotiation successful
0925 07:28:16.141683 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260925 07:28:16.142058 21349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:16.132846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.65:44875)
0925 07:28:16.132983 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:16.133026 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 07:28:16.133564 (+ 538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:28:16.133829 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:28:16.133833 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:28:16.134387 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:28:16.134561 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:28:16.134565 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:28:16.135303 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:28:16.135305 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:28:16.136090 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:28:16.136092 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:28:16.136532 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:28:16.136533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:28:16.136647 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:28:16.137018 (+ 371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:28:16.137025 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:28:16.138579 (+ 1554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:28:16.140347 (+ 1768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:28:16.140355 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:28:16.140357 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:28:16.140668 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:28:16.140876 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:28:16.140877 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:28:16.140877 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:28:16.141016 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:28:16.141313 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:28:16.141318 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:28:16.141493 (+ 175us) client_negotiation.cc:769] Sending connection context
0925 07:28:16.141605 (+ 112us) client_negotiation.cc:240] Negotiation successful
0925 07:28:16.141765 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":67,"threads_started":1}
I20260925 07:28:16.142572 21347 heartbeater.cc:347] Connected to a master server at 127.5.158.126:32843
I20260925 07:28:16.142669 21347 heartbeater.cc:464] Registering TS with master...
I20260925 07:28:16.142853 21347 heartbeater.cc:511] Master 127.5.158.126:32843 requested a full tablet report, sending...
I20260925 07:28:16.143309 21156 ts_manager.cc:194] Registered new tserver with Master: 3d9933f641294109bd395addc5ab8c70 (127.5.158.65:37045)
I20260925 07:28:16.143941 21156 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.65@KRBTEST.COM'} at 127.5.158.65:44875
WARNING: no policy specified for kudu/127.5.158.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab.
Entry for principal kudu/127.5.158.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab.
I20260925 07:28:16.162057 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.66:0
--local_ip_for_outbound_sockets=127.5.158.66
--webserver_interface=127.5.158.66
--webserver_port=0
--tserver_master_addrs=127.5.158.126:32843
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab
--principal=kudu/127.5.158.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:44845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:28:16.232774 21352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:28:16.232959 21352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:28:16.232992 21352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:28:16.234037 21352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:28:16.234100 21352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:28:16.234141 21352 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.66
I20260925 07:28:16.235853 21352 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:44845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab
--principal=kudu/127.5.158.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.158.66
--webserver_port=0
--tserver_master_addrs=127.5.158.126:32843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.66
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:28:16.236084 21352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:28:16.236285 21352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:28:16.240875 21357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:28:16.240912 21358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:28:16.240929 21360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:28:16.241009 21352 server_base.cc:1034] running on GCE node
I20260925 07:28:16.241235 21352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:28:16.241474 21352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:28:16.242625 21352 hybrid_clock.cc:648] HybridClock initialized: now 1790321296242614 us; error 38 us; skew 500 ppm
Sep 25 07:28:16 dist-test-slave-1drq krb5kdc[21111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321296, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:28:16.244735 21352 init.cc:377] Logged in from keytab as kudu/127.5.158.66@KRBTEST.COM (short username kudu)
I20260925 07:28:16.245340 21352 webserver.cc:460] Webserver started at http://127.5.158.66:45961/ using document root <none> and password file <none>
I20260925 07:28:16.245594 21352 fs_manager.cc:359] Metadata directory not provided
I20260925 07:28:16.245666 21352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:28:16.245774 21352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:28:16.246685 21352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-1/data/instance:
uuid: "98d5e5d89d654dab96fc59e589ef6cfd"
format_stamp: "Formatted at 2026-09-25 07:28:16 on dist-test-slave-1drq"
server_key: "ed675f764ddbc1fc0df3b050bb9dfc3a"
server_key_iv: "38b84b7c338f9ff97f9466b778bd7b49"
server_key_version: "encryptionkey@0"
I20260925 07:28:16.246973 21352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-1/wal/instance:
uuid: "98d5e5d89d654dab96fc59e589ef6cfd"
format_stamp: "Formatted at 2026-09-25 07:28:16 on dist-test-slave-1drq"
server_key: "ed675f764ddbc1fc0df3b050bb9dfc3a"
server_key_iv: "38b84b7c338f9ff97f9466b778bd7b49"
server_key_version: "encryptionkey@0"
I20260925 07:28:16.248112 21352 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 07:28:16.248759 21367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:28:16.248958 21352 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:28:16.249054 21352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-1/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-1/wal
uuid: "98d5e5d89d654dab96fc59e589ef6cfd"
format_stamp: "Formatted at 2026-09-25 07:28:16 on dist-test-slave-1drq"
server_key: "ed675f764ddbc1fc0df3b050bb9dfc3a"
server_key_iv: "38b84b7c338f9ff97f9466b778bd7b49"
server_key_version: "encryptionkey@0"
I20260925 07:28:16.249137 21352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:28:16.272795 21352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:28:16.274355 21352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:28:16.274503 21352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:28:16.274699 21352 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:28:16.274972 21352 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:28:16.275024 21352 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:28:16.275056 21352 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:28:16.275082 21352 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:28:16.280408 21352 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.66:43065
I20260925 07:28:16.280499 21480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.66:43065 every 8 connection(s)
I20260925 07:28:16.280741 21352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
I20260925 07:28:16.285630 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 21352
I20260925 07:28:16.285705 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-1/wal/instance
I20260925 07:28:16.285884 5753 external_mini_cluster.cc:1428] Setting key c74d755c67f1ebd627d99a7a91b7d610
Sep 25 07:28:16 dist-test-slave-1drq krb5kdc[21111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321296, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.66@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:28:16.288911 21196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:16.281385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:44247 (local address 127.5.158.126:32843)
0925 07:28:16.281423 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 07:28:16.281425 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:28:16.281597 (+ 172us) server_negotiation.cc:407] Connection header received
0925 07:28:16.282002 (+ 405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:28:16.282003 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:28:16.282012 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:28:16.282056 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:28:16.282837 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:28:16.283289 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:28:16.284100 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:28:16.284321 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:28:16.284387 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:28:16.286392 (+ 2005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:28:16.286393 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:28:16.286394 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:28:16.286401 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:28:16.287664 (+ 1263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:28:16.288038 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:28:16.288040 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:28:16.288040 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:28:16.288095 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:28:16.288322 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:28:16.288323 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:28:16.288323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:28:16.288427 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:28:16.288501 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 07:28:16.288784 (+ 283us) server_negotiation.cc:299] Negotiation successful
0925 07:28:16.288851 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 07:28:16.289170 21483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:16.281346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.66:44247)
0925 07:28:16.281483 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:16.281536 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0925 07:28:16.281923 (+ 387us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:28:16.282105 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:28:16.282107 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:28:16.282594 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:28:16.282763 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:28:16.282767 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:28:16.283359 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:28:16.283360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:28:16.284027 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:28:16.284029 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:28:16.284403 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:28:16.284404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:28:16.284501 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:28:16.284913 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:28:16.284919 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:28:16.286307 (+ 1388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:28:16.287756 (+ 1449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:28:16.287762 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:28:16.287763 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:28:16.287957 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:28:16.288155 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:28:16.288156 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:28:16.288156 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:28:16.288255 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:28:16.288493 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:28:16.288498 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:28:16.288635 (+ 137us) client_negotiation.cc:769] Sending connection context
0925 07:28:16.288751 (+ 116us) client_negotiation.cc:240] Negotiation successful
0925 07:28:16.288902 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":73,"threads_started":1}
I20260925 07:28:16.289629 21481 heartbeater.cc:347] Connected to a master server at 127.5.158.126:32843
I20260925 07:28:16.289717 21481 heartbeater.cc:464] Registering TS with master...
I20260925 07:28:16.289871 21481 heartbeater.cc:511] Master 127.5.158.126:32843 requested a full tablet report, sending...
I20260925 07:28:16.290203 21156 ts_manager.cc:194] Registered new tserver with Master: 98d5e5d89d654dab96fc59e589ef6cfd (127.5.158.66:43065)
I20260925 07:28:16.290702 21156 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.66@KRBTEST.COM'} at 127.5.158.66:44247
WARNING: no policy specified for kudu/127.5.158.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab.
Entry for principal kudu/127.5.158.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab.
I20260925 07:28:16.309397 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.67:0
--local_ip_for_outbound_sockets=127.5.158.67
--webserver_interface=127.5.158.67
--webserver_port=0
--tserver_master_addrs=127.5.158.126:32843
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab
--principal=kudu/127.5.158.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:44845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:28:16.382078 21486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:28:16.382302 21486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:28:16.382344 21486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:28:16.383430 21486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:28:16.383494 21486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:28:16.383538 21486 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.67
I20260925 07:28:16.384912 21486 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:44845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab
--principal=kudu/127.5.158.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.158.67
--webserver_port=0
--tserver_master_addrs=127.5.158.126:32843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21486
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.67
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:28:16.385179 21486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:28:16.385408 21486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:28:16.389732 21491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:28:16.389746 21494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:28:16.389882 21486 server_base.cc:1034] running on GCE node
W20260925 07:28:16.389813 21492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:28:16.390125 21486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:28:16.390339 21486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:28:16.391502 21486 hybrid_clock.cc:648] HybridClock initialized: now 1790321296391491 us; error 21 us; skew 500 ppm
Sep 25 07:28:16 dist-test-slave-1drq krb5kdc[21111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321296, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:28:16.393911 21486 init.cc:377] Logged in from keytab as kudu/127.5.158.67@KRBTEST.COM (short username kudu)
I20260925 07:28:16.394320 21486 webserver.cc:460] Webserver started at http://127.5.158.67:38377/ using document root <none> and password file <none>
I20260925 07:28:16.394541 21486 fs_manager.cc:359] Metadata directory not provided
I20260925 07:28:16.394609 21486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:28:16.394762 21486 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:28:16.395735 21486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-2/data/instance:
uuid: "6d397540b57a4663935e8d3c4f5a417e"
format_stamp: "Formatted at 2026-09-25 07:28:16 on dist-test-slave-1drq"
server_key: "38f38507c7e2c77c84af8fd08ed34aa4"
server_key_iv: "2704311d537a11eb8df9d149c48cce82"
server_key_version: "encryptionkey@0"
I20260925 07:28:16.396093 21486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-2/wal/instance:
uuid: "6d397540b57a4663935e8d3c4f5a417e"
format_stamp: "Formatted at 2026-09-25 07:28:16 on dist-test-slave-1drq"
server_key: "38f38507c7e2c77c84af8fd08ed34aa4"
server_key_iv: "2704311d537a11eb8df9d149c48cce82"
server_key_version: "encryptionkey@0"
I20260925 07:28:16.397358 21486 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 07:28:16.397975 21501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:28:16.398114 21486 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 07:28:16.398201 21486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-2/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-2/wal
uuid: "6d397540b57a4663935e8d3c4f5a417e"
format_stamp: "Formatted at 2026-09-25 07:28:16 on dist-test-slave-1drq"
server_key: "38f38507c7e2c77c84af8fd08ed34aa4"
server_key_iv: "2704311d537a11eb8df9d149c48cce82"
server_key_version: "encryptionkey@0"
I20260925 07:28:16.398293 21486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:28:16.406061 21486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:28:16.407963 21486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:28:16.408100 21486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:28:16.408308 21486 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:28:16.408565 21486 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:28:16.408610 21486 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:28:16.408643 21486 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:28:16.408666 21486 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:28:16.413992 21486 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.67:45323
I20260925 07:28:16.414096 21614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.67:45323 every 8 connection(s)
I20260925 07:28:16.414338 21486 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
Sep 25 07:28:16 dist-test-slave-1drq krb5kdc[21111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321296, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.67@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:28:16.422061 21196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:16.414929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:36049 (local address 127.5.158.126:32843)
0925 07:28:16.414993 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0925 07:28:16.414995 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:28:16.415166 (+ 171us) server_negotiation.cc:407] Connection header received
0925 07:28:16.415570 (+ 404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:28:16.415571 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:28:16.415580 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:28:16.415624 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:28:16.416511 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:28:16.416990 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:28:16.417851 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:28:16.418013 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:28:16.418059 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:28:16.419782 (+ 1723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:28:16.419784 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:28:16.419784 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:28:16.419790 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:28:16.421051 (+ 1261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:28:16.421436 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:28:16.421436 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:28:16.421437 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:28:16.421481 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:28:16.421651 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:28:16.421651 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:28:16.421651 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:28:16.421734 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:28:16.421780 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0925 07:28:16.421958 (+ 178us) server_negotiation.cc:299] Negotiation successful
0925 07:28:16.422008 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260925 07:28:16.422173 21617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:16.414902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.67:36049)
0925 07:28:16.415055 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:16.415086 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0925 07:28:16.415466 (+ 380us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:28:16.415687 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:28:16.415690 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:28:16.416270 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:28:16.416432 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:28:16.416435 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:28:16.417061 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:28:16.417062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:28:16.417785 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:28:16.417787 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:28:16.418074 (+ 287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:28:16.418075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:28:16.418153 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:28:16.418547 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:28:16.418554 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:28:16.419708 (+ 1154us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:28:16.421209 (+ 1501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:28:16.421213 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:28:16.421214 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:28:16.421381 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:28:16.421530 (+ 149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:28:16.421531 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:28:16.421531 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:28:16.421606 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:28:16.421782 (+ 176us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:28:16.421785 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:28:16.421867 (+ 82us) client_negotiation.cc:769] Sending connection context
0925 07:28:16.421938 (+ 71us) client_negotiation.cc:240] Negotiation successful
0925 07:28:16.422027 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":93,"threads_started":1}
I20260925 07:28:16.422582 21615 heartbeater.cc:347] Connected to a master server at 127.5.158.126:32843
I20260925 07:28:16.422668 21615 heartbeater.cc:464] Registering TS with master...
I20260925 07:28:16.422817 21615 heartbeater.cc:511] Master 127.5.158.126:32843 requested a full tablet report, sending...
I20260925 07:28:16.423146 21156 ts_manager.cc:194] Registered new tserver with Master: 6d397540b57a4663935e8d3c4f5a417e (127.5.158.67:45323)
I20260925 07:28:16.423200 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 21486
I20260925 07:28:16.423308 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-2/wal/instance
I20260925 07:28:16.423480 5753 external_mini_cluster.cc:1428] Setting key 12d9af2dedc8ed56ae85a5faa4f9608e
I20260925 07:28:16.423554 21156 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.67@KRBTEST.COM'} at 127.5.158.67:36049
I20260925 07:28:16.424142 5753 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 07:28:16.430527 21196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:16.425452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54816 (local address 127.5.158.126:32843)
0925 07:28:16.425549 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0925 07:28:16.425551 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:28:16.425622 (+ 71us) server_negotiation.cc:407] Connection header received
0925 07:28:16.425700 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:28:16.425701 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:28:16.425709 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:28:16.425754 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:28:16.426307 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:28:16.426714 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:28:16.427382 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:28:16.427560 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:28:16.427621 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:28:16.428518 (+ 897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:28:16.428519 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:28:16.428520 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:28:16.428526 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:28:16.429565 (+ 1039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:28:16.429852 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:28:16.429853 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:28:16.429854 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:28:16.429902 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:28:16.430074 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:28:16.430074 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:28:16.430075 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:28:16.430194 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:28:16.430283 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0925 07:28:16.430376 (+ 93us) server_negotiation.cc:299] Negotiation successful
0925 07:28:16.430449 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260925 07:28:16.431738 21156 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54816:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 07:28:16.433245 21156 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 07:28:16.438776 21623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:16.435057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.67:45323 (local address 127.0.0.1:42102)
0925 07:28:16.435414 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:16.435448 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 07:28:16.435525 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:28:16.435721 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:28:16.435724 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:28:16.435726 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:28:16.435794 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:28:16.435796 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:28:16.436984 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:28:16.436985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:28:16.437773 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:28:16.437776 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:28:16.438512 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:28:16.438513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:28:16.438599 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:28:16.438611 (+ 12us) client_negotiation.cc:769] Sending connection context
0925 07:28:16.438649 (+ 38us) client_negotiation.cc:240] Negotiation successful
0925 07:28:16.438689 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":47,"threads_started":1}
I20260925 07:28:16.438850 21626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:16.435111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33086 (local address 127.5.158.65:37045)
0925 07:28:16.435427 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0925 07:28:16.435430 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 07:28:16.435649 (+ 219us) server_negotiation.cc:407] Connection header received
0925 07:28:16.435687 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:28:16.435689 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:28:16.435703 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:28:16.435847 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:28:16.436051 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:28:16.437007 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:28:16.438111 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:28:16.438510 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:28:16.438617 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:28:16.438669 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 07:28:16.438745 (+ 76us) server_negotiation.cc:299] Negotiation successful
0925 07:28:16.438782 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":86,"threads_started":1}
I20260925 07:28:16.438861 21625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:16.435019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:37045 (local address 127.0.0.1:33086)
0925 07:28:16.435597 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:16.435602 (+ 5us) client_negotiation.cc:174] Beginning negotiation
0925 07:28:16.435629 (+ 27us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:28:16.435929 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:28:16.435931 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:28:16.435933 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:28:16.435975 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:28:16.435977 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:28:16.437119 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:28:16.437120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:28:16.438044 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:28:16.438045 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:28:16.438620 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:28:16.438621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:28:16.438678 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:28:16.438682 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 07:28:16.438733 (+ 51us) client_negotiation.cc:240] Negotiation successful
0925 07:28:16.438769 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":61,"threads_started":1}
I20260925 07:28:16.439155 21628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:16.435255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43610 (local address 127.5.158.66:43065)
0925 07:28:16.435390 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0925 07:28:16.435394 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 07:28:16.435736 (+ 342us) server_negotiation.cc:407] Connection header received
0925 07:28:16.435779 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:28:16.435781 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:28:16.435795 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:28:16.435855 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:28:16.436038 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:28:16.436657 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:28:16.438041 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:28:16.438706 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:28:16.438762 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:28:16.438809 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 07:28:16.439039 (+ 230us) server_negotiation.cc:299] Negotiation successful
0925 07:28:16.439079 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":76,"threads_started":1}
I20260925 07:28:16.439244 21627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:16.435233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42102 (local address 127.5.158.67:45323)
0925 07:28:16.435385 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0925 07:28:16.435389 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 07:28:16.435531 (+ 142us) server_negotiation.cc:407] Connection header received
0925 07:28:16.435590 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:28:16.435591 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:28:16.435601 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:28:16.435863 (+ 262us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:28:16.435914 (+ 51us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:28:16.436883 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:28:16.437861 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:28:16.438426 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:28:16.439064 (+ 638us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:28:16.439113 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0925 07:28:16.439133 (+ 20us) server_negotiation.cc:299] Negotiation successful
0925 07:28:16.439177 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":97,"threads_started":1}
I20260925 07:28:16.439114 21624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:16.435057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.66:43065 (local address 127.0.0.1:43610)
0925 07:28:16.435673 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:16.435678 (+ 5us) client_negotiation.cc:174] Beginning negotiation
0925 07:28:16.435728 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:28:16.435928 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:28:16.435928 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:28:16.435930 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:28:16.435973 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:28:16.435974 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:28:16.436781 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:28:16.436782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:28:16.437977 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:28:16.437978 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:28:16.438841 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:28:16.438842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:28:16.438909 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:28:16.438913 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 07:28:16.438962 (+ 49us) client_negotiation.cc:240] Negotiation successful
0925 07:28:16.439002 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"mutex_wait_us":11,"thread_start_us":73,"threads_started":1}
I20260925 07:28:16.439839 21549 tablet_service.cc:1467] Processing CreateTablet for tablet 7b54963a844341f89e7c3fabd41452ae (DEFAULT_TABLE table=test-table [id=98e9e0d383d64581852531f8450acfa8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:28:16.439950 21415 tablet_service.cc:1467] Processing CreateTablet for tablet 7b54963a844341f89e7c3fabd41452ae (DEFAULT_TABLE table=test-table [id=98e9e0d383d64581852531f8450acfa8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:28:16.440165 21549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b54963a844341f89e7c3fabd41452ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:28:16.440313 21415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b54963a844341f89e7c3fabd41452ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:28:16.441498 21281 tablet_service.cc:1467] Processing CreateTablet for tablet 7b54963a844341f89e7c3fabd41452ae (DEFAULT_TABLE table=test-table [id=98e9e0d383d64581852531f8450acfa8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:28:16.441782 21281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b54963a844341f89e7c3fabd41452ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:28:16.442466 21629 tablet_bootstrap.cc:492] T 7b54963a844341f89e7c3fabd41452ae P 6d397540b57a4663935e8d3c4f5a417e: Bootstrap starting.
I20260925 07:28:16.443095 21629 tablet_bootstrap.cc:654] T 7b54963a844341f89e7c3fabd41452ae P 6d397540b57a4663935e8d3c4f5a417e: Neither blocks nor log segments found. Creating new log.
I20260925 07:28:16.443199 21630 tablet_bootstrap.cc:492] T 7b54963a844341f89e7c3fabd41452ae P 98d5e5d89d654dab96fc59e589ef6cfd: Bootstrap starting.
I20260925 07:28:16.443693 21629 log.cc:824] T 7b54963a844341f89e7c3fabd41452ae P 6d397540b57a4663935e8d3c4f5a417e: Log is configured to *not* fsync() on all Append() calls
I20260925 07:28:16.443838 21630 tablet_bootstrap.cc:654] T 7b54963a844341f89e7c3fabd41452ae P 98d5e5d89d654dab96fc59e589ef6cfd: Neither blocks nor log segments found. Creating new log.
I20260925 07:28:16.444350 21630 log.cc:824] T 7b54963a844341f89e7c3fabd41452ae P 98d5e5d89d654dab96fc59e589ef6cfd: Log is configured to *not* fsync() on all Append() calls
I20260925 07:28:16.444482 21629 tablet_bootstrap.cc:492] T 7b54963a844341f89e7c3fabd41452ae P 6d397540b57a4663935e8d3c4f5a417e: No bootstrap required, opened a new log
I20260925 07:28:16.444554 21629 ts_tablet_manager.cc:1397] T 7b54963a844341f89e7c3fabd41452ae P 6d397540b57a4663935e8d3c4f5a417e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 07:28:16.444718 21633 tablet_bootstrap.cc:492] T 7b54963a844341f89e7c3fabd41452ae P 3d9933f641294109bd395addc5ab8c70: Bootstrap starting.
I20260925 07:28:16.445633 21630 tablet_bootstrap.cc:492] T 7b54963a844341f89e7c3fabd41452ae P 98d5e5d89d654dab96fc59e589ef6cfd: No bootstrap required, opened a new log
I20260925 07:28:16.445717 21630 ts_tablet_manager.cc:1397] T 7b54963a844341f89e7c3fabd41452ae P 98d5e5d89d654dab96fc59e589ef6cfd: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260925 07:28:16.446681 21629 raft_consensus.cc:348] T 7b54963a844341f89e7c3fabd41452ae P 6d397540b57a4663935e8d3c4f5a417e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d9933f641294109bd395addc5ab8c70" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 37045 } } peers { permanent_uuid: "98d5e5d89d654dab96fc59e589ef6cfd" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 43065 } } peers { permanent_uuid: "6d397540b57a4663935e8d3c4f5a417e" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 45323 } }
I20260925 07:28:16.446842 21629 raft_consensus.cc:374] T 7b54963a844341f89e7c3fabd41452ae P 6d397540b57a4663935e8d3c4f5a417e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:28:16.446877 21629 raft_consensus.cc:729] T 7b54963a844341f89e7c3fabd41452ae P 6d397540b57a4663935e8d3c4f5a417e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d397540b57a4663935e8d3c4f5a417e, State: Initialized, Role: FOLLOWER
I20260925 07:28:16.447002 21629 consensus_queue.cc:260] T 7b54963a844341f89e7c3fabd41452ae P 6d397540b57a4663935e8d3c4f5a417e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d9933f641294109bd395addc5ab8c70" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 37045 } } peers { permanent_uuid: "98d5e5d89d654dab96fc59e589ef6cfd" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 43065 } } peers { permanent_uuid: "6d397540b57a4663935e8d3c4f5a417e" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 45323 } }
I20260925 07:28:16.447302 21629 ts_tablet_manager.cc:1428] T 7b54963a844341f89e7c3fabd41452ae P 6d397540b57a4663935e8d3c4f5a417e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 07:28:16.447444 21615 heartbeater.cc:503] Master 127.5.158.126:32843 was elected leader, sending a full tablet report...
I20260925 07:28:16.447732 21630 raft_consensus.cc:348] T 7b54963a844341f89e7c3fabd41452ae P 98d5e5d89d654dab96fc59e589ef6cfd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d9933f641294109bd395addc5ab8c70" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 37045 } } peers { permanent_uuid: "98d5e5d89d654dab96fc59e589ef6cfd" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 43065 } } peers { permanent_uuid: "6d397540b57a4663935e8d3c4f5a417e" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 45323 } }
I20260925 07:28:16.447896 21630 raft_consensus.cc:374] T 7b54963a844341f89e7c3fabd41452ae P 98d5e5d89d654dab96fc59e589ef6cfd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:28:16.447919 21630 raft_consensus.cc:729] T 7b54963a844341f89e7c3fabd41452ae P 98d5e5d89d654dab96fc59e589ef6cfd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98d5e5d89d654dab96fc59e589ef6cfd, State: Initialized, Role: FOLLOWER
I20260925 07:28:16.448048 21630 consensus_queue.cc:260] T 7b54963a844341f89e7c3fabd41452ae P 98d5e5d89d654dab96fc59e589ef6cfd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d9933f641294109bd395addc5ab8c70" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 37045 } } peers { permanent_uuid: "98d5e5d89d654dab96fc59e589ef6cfd" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 43065 } } peers { permanent_uuid: "6d397540b57a4663935e8d3c4f5a417e" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 45323 } }
I20260925 07:28:16.448321 21630 ts_tablet_manager.cc:1428] T 7b54963a844341f89e7c3fabd41452ae P 98d5e5d89d654dab96fc59e589ef6cfd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 07:28:16.448324 21481 heartbeater.cc:503] Master 127.5.158.126:32843 was elected leader, sending a full tablet report...
I20260925 07:28:16.448451 21633 tablet_bootstrap.cc:654] T 7b54963a844341f89e7c3fabd41452ae P 3d9933f641294109bd395addc5ab8c70: Neither blocks nor log segments found. Creating new log.
I20260925 07:28:16.449016 21633 log.cc:824] T 7b54963a844341f89e7c3fabd41452ae P 3d9933f641294109bd395addc5ab8c70: Log is configured to *not* fsync() on all Append() calls
I20260925 07:28:16.449685 21633 tablet_bootstrap.cc:492] T 7b54963a844341f89e7c3fabd41452ae P 3d9933f641294109bd395addc5ab8c70: No bootstrap required, opened a new log
I20260925 07:28:16.449736 21633 ts_tablet_manager.cc:1397] T 7b54963a844341f89e7c3fabd41452ae P 3d9933f641294109bd395addc5ab8c70: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20260925 07:28:16.451118 21633 raft_consensus.cc:348] T 7b54963a844341f89e7c3fabd41452ae P 3d9933f641294109bd395addc5ab8c70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d9933f641294109bd395addc5ab8c70" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 37045 } } peers { permanent_uuid: "98d5e5d89d654dab96fc59e589ef6cfd" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 43065 } } peers { permanent_uuid: "6d397540b57a4663935e8d3c4f5a417e" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 45323 } }
I20260925 07:28:16.451272 21633 raft_consensus.cc:374] T 7b54963a844341f89e7c3fabd41452ae P 3d9933f641294109bd395addc5ab8c70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:28:16.451295 21633 raft_consensus.cc:729] T 7b54963a844341f89e7c3fabd41452ae P 3d9933f641294109bd395addc5ab8c70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d9933f641294109bd395addc5ab8c70, State: Initialized, Role: FOLLOWER
I20260925 07:28:16.451414 21633 consensus_queue.cc:260] T 7b54963a844341f89e7c3fabd41452ae P 3d9933f641294109bd395addc5ab8c70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d9933f641294109bd395addc5ab8c70" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 37045 } } peers { permanent_uuid: "98d5e5d89d654dab96fc59e589ef6cfd" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 43065 } } peers { permanent_uuid: "6d397540b57a4663935e8d3c4f5a417e" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 45323 } }
I20260925 07:28:16.451673 21633 ts_tablet_manager.cc:1428] T 7b54963a844341f89e7c3fabd41452ae P 3d9933f641294109bd395addc5ab8c70: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260925 07:28:16.451707 21347 heartbeater.cc:503] Master 127.5.158.126:32843 was elected leader, sending a full tablet report...
W20260925 07:28:16.531255 21482 tablet.cc:2369] T 7b54963a844341f89e7c3fabd41452ae P 98d5e5d89d654dab96fc59e589ef6cfd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 07:28:16.555881 21637 raft_consensus.cc:482] T 7b54963a844341f89e7c3fabd41452ae P 3d9933f641294109bd395addc5ab8c70 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 07:28:16.555997 21637 raft_consensus.cc:504] T 7b54963a844341f89e7c3fabd41452ae P 3d9933f641294109bd395addc5ab8c70 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d9933f641294109bd395addc5ab8c70" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 37045 } } peers { permanent_uuid: "98d5e5d89d654dab96fc59e589ef6cfd" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 43065 } } peers { permanent_uuid: "6d397540b57a4663935e8d3c4f5a417e" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 45323 } }
I20260925 07:28:16.556360 21637 leader_election.cc:288] T 7b54963a844341f89e7c3fabd41452ae P 3d9933f641294109bd395addc5ab8c70 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98d5e5d89d654dab96fc59e589ef6cfd (127.5.158.66:43065), 6d397540b57a4663935e8d3c4f5a417e (127.5.158.67:45323)
I20260925 07:28:16.559278 21349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:16.556526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.66:43065 (local address 127.5.158.65:60213)
0925 07:28:16.556562 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:16.556571 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 07:28:16.556623 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:28:16.556761 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:28:16.556762 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:28:16.556764 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:28:16.556841 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:28:16.556842 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:28:16.557575 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:28:16.557576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:28:16.558457 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:28:16.558459 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:28:16.559026 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:28:16.559027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:28:16.559106 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:28:16.559109 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 07:28:16.559169 (+ 60us) client_negotiation.cc:240] Negotiation successful
0925 07:28:16.559209 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260925 07:28:16.559337 21628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:16.556570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:60213 (local address 127.5.158.66:43065)
0925 07:28:16.556650 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0925 07:28:16.556651 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:28:16.556658 (+ 7us) server_negotiation.cc:407] Connection header received
0925 07:28:16.556679 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:28:16.556680 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:28:16.556687 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:28:16.556734 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:28:16.556914 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:28:16.557503 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:28:16.558525 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:28:16.558944 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:28:16.559009 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:28:16.559015 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0925 07:28:16.559189 (+ 174us) server_negotiation.cc:299] Negotiation successful
0925 07:28:16.559223 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260925 07:28:16.559481 21638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:16.556526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.67:45323 (local address 127.5.158.65:54611)
0925 07:28:16.556844 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:16.556852 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 07:28:16.556918 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:28:16.557024 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:28:16.557024 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:28:16.557025 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:28:16.557103 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:28:16.557105 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:28:16.557823 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:28:16.557823 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:28:16.558621 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:28:16.558623 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:28:16.559193 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:28:16.559194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:28:16.559259 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:28:16.559263 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 07:28:16.559348 (+ 85us) client_negotiation.cc:240] Negotiation successful
0925 07:28:16.559392 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"mutex_wait_us":141,"thread_start_us":107,"threads_started":1}
I20260925 07:28:16.559494 21627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:16.556569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:54611 (local address 127.5.158.67:45323)
0925 07:28:16.556634 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0925 07:28:16.556636 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:28:16.556923 (+ 287us) server_negotiation.cc:407] Connection header received
0925 07:28:16.556973 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:28:16.556974 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:28:16.556981 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:28:16.557018 (+ 37us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:28:16.557202 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:28:16.557746 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:28:16.558689 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:28:16.559119 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:28:16.559178 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:28:16.559185 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 07:28:16.559368 (+ 183us) server_negotiation.cc:299] Negotiation successful
0925 07:28:16.559411 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 07:28:16.559621 21435 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b54963a844341f89e7c3fabd41452ae" candidate_uuid: "3d9933f641294109bd395addc5ab8c70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98d5e5d89d654dab96fc59e589ef6cfd" is_pre_election: true
I20260925 07:28:16.559764 21435 raft_consensus.cc:2454] T 7b54963a844341f89e7c3fabd41452ae P 98d5e5d89d654dab96fc59e589ef6cfd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d9933f641294109bd395addc5ab8c70 in term 0.
I20260925 07:28:16.559839 21569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b54963a844341f89e7c3fabd41452ae" candidate_uuid: "3d9933f641294109bd395addc5ab8c70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d397540b57a4663935e8d3c4f5a417e" is_pre_election: true
I20260925 07:28:16.559965 21569 raft_consensus.cc:2454] T 7b54963a844341f89e7c3fabd41452ae P 6d397540b57a4663935e8d3c4f5a417e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d9933f641294109bd395addc5ab8c70 in term 0.
I20260925 07:28:16.559993 21235 leader_election.cc:302] T 7b54963a844341f89e7c3fabd41452ae P 3d9933f641294109bd395addc5ab8c70 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d9933f641294109bd395addc5ab8c70, 98d5e5d89d654dab96fc59e589ef6cfd; no voters:
I20260925 07:28:16.560150 21637 raft_consensus.cc:2781] T 7b54963a844341f89e7c3fabd41452ae P 3d9933f641294109bd395addc5ab8c70 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 07:28:16.560205 21637 raft_consensus.cc:482] T 7b54963a844341f89e7c3fabd41452ae P 3d9933f641294109bd395addc5ab8c70 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 07:28:16.560221 21637 raft_consensus.cc:3037] T 7b54963a844341f89e7c3fabd41452ae P 3d9933f641294109bd395addc5ab8c70 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:28:16.560907 21637 raft_consensus.cc:504] T 7b54963a844341f89e7c3fabd41452ae P 3d9933f641294109bd395addc5ab8c70 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d9933f641294109bd395addc5ab8c70" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 37045 } } peers { permanent_uuid: "98d5e5d89d654dab96fc59e589ef6cfd" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 43065 } } peers { permanent_uuid: "6d397540b57a4663935e8d3c4f5a417e" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 45323 } }
I20260925 07:28:16.561028 21637 leader_election.cc:288] T 7b54963a844341f89e7c3fabd41452ae P 3d9933f641294109bd395addc5ab8c70 [CANDIDATE]: Term 1 election: Requested vote from peers 98d5e5d89d654dab96fc59e589ef6cfd (127.5.158.66:43065), 6d397540b57a4663935e8d3c4f5a417e (127.5.158.67:45323)
I20260925 07:28:16.561260 21435 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b54963a844341f89e7c3fabd41452ae" candidate_uuid: "3d9933f641294109bd395addc5ab8c70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98d5e5d89d654dab96fc59e589ef6cfd"
I20260925 07:28:16.561276 21569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b54963a844341f89e7c3fabd41452ae" candidate_uuid: "3d9933f641294109bd395addc5ab8c70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d397540b57a4663935e8d3c4f5a417e"
I20260925 07:28:16.561336 21569 raft_consensus.cc:3037] T 7b54963a844341f89e7c3fabd41452ae P 6d397540b57a4663935e8d3c4f5a417e [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:28:16.561336 21435 raft_consensus.cc:3037] T 7b54963a844341f89e7c3fabd41452ae P 98d5e5d89d654dab96fc59e589ef6cfd [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:28:16.562191 21569 raft_consensus.cc:2454] T 7b54963a844341f89e7c3fabd41452ae P 6d397540b57a4663935e8d3c4f5a417e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d9933f641294109bd395addc5ab8c70 in term 1.
I20260925 07:28:16.562191 21435 raft_consensus.cc:2454] T 7b54963a844341f89e7c3fabd41452ae P 98d5e5d89d654dab96fc59e589ef6cfd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d9933f641294109bd395addc5ab8c70 in term 1.
I20260925 07:28:16.562348 21235 leader_election.cc:302] T 7b54963a844341f89e7c3fabd41452ae P 3d9933f641294109bd395addc5ab8c70 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d9933f641294109bd395addc5ab8c70, 98d5e5d89d654dab96fc59e589ef6cfd; no voters:
I20260925 07:28:16.562428 21637 raft_consensus.cc:2781] T 7b54963a844341f89e7c3fabd41452ae P 3d9933f641294109bd395addc5ab8c70 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:28:16.562551 21637 raft_consensus.cc:686] T 7b54963a844341f89e7c3fabd41452ae P 3d9933f641294109bd395addc5ab8c70 [term 1 LEADER]: Becoming Leader. State: Replica: 3d9933f641294109bd395addc5ab8c70, State: Running, Role: LEADER
I20260925 07:28:16.562647 21637 consensus_queue.cc:237] T 7b54963a844341f89e7c3fabd41452ae P 3d9933f641294109bd395addc5ab8c70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d9933f641294109bd395addc5ab8c70" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 37045 } } peers { permanent_uuid: "98d5e5d89d654dab96fc59e589ef6cfd" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 43065 } } peers { permanent_uuid: "6d397540b57a4663935e8d3c4f5a417e" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 45323 } }
I20260925 07:28:16.563409 21156 catalog_manager.cc:5495] T 7b54963a844341f89e7c3fabd41452ae P 3d9933f641294109bd395addc5ab8c70 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d9933f641294109bd395addc5ab8c70 (127.5.158.65). New cstate: current_term: 1 leader_uuid: "3d9933f641294109bd395addc5ab8c70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d9933f641294109bd395addc5ab8c70" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 37045 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98d5e5d89d654dab96fc59e589ef6cfd" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 43065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d397540b57a4663935e8d3c4f5a417e" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 45323 } health_report { overall_health: UNKNOWN } } }
I20260925 07:28:16.582068 21626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:16.579991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33096 (local address 127.5.158.65:37045)
0925 07:28:16.580068 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0925 07:28:16.580069 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:28:16.580077 (+ 8us) server_negotiation.cc:407] Connection header received
0925 07:28:16.580107 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:28:16.580107 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:28:16.580127 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:28:16.580196 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 07:28:16.580318 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:28:16.580703 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:28:16.581423 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:28:16.581587 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:28:16.581647 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:28:16.581785 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 07:28:16.581869 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 07:28:16.581941 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 07:28:16.581978 (+ 37us) server_negotiation.cc:299] Negotiation successful
0925 07:28:16.581995 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260925 07:28:16.583447 21569 raft_consensus.cc:1261] T 7b54963a844341f89e7c3fabd41452ae P 6d397540b57a4663935e8d3c4f5a417e [term 1 FOLLOWER]: Refusing update from remote peer 3d9933f641294109bd395addc5ab8c70: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:28:16.583521 21435 raft_consensus.cc:1261] T 7b54963a844341f89e7c3fabd41452ae P 98d5e5d89d654dab96fc59e589ef6cfd [term 1 FOLLOWER]: Refusing update from remote peer 3d9933f641294109bd395addc5ab8c70: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:28:16.583775 21637 consensus_queue.cc:1035] T 7b54963a844341f89e7c3fabd41452ae P 3d9933f641294109bd395addc5ab8c70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d397540b57a4663935e8d3c4f5a417e" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 45323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:28:16.583897 21639 consensus_queue.cc:1035] T 7b54963a844341f89e7c3fabd41452ae P 3d9933f641294109bd395addc5ab8c70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98d5e5d89d654dab96fc59e589ef6cfd" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 43065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:28:16.587324 21644 mvcc.cc:205] Tried to move back new op lower bound from 7333156030803759104 to 7333156030721204224. Current Snapshot: MvccSnapshot[applied={T|T < 7333156030803759104 or (T in {7333156030803759104})}]
I20260925 07:28:16.588477 21155 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54816:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 07:28:16.588553 21155 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54816:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 07:28:16.589222 21155 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de: Sending DeleteTablet for 3 replicas of tablet 7b54963a844341f89e7c3fabd41452ae
I20260925 07:28:16.589499 21281 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b54963a844341f89e7c3fabd41452ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:28:16 UTC) from {username='kudu', principal='kudu/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:33086
I20260925 07:28:16.589527 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 21218
I20260925 07:28:16.589532 21415 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b54963a844341f89e7c3fabd41452ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:28:16 UTC) from {username='kudu', principal='kudu/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:43610
I20260925 07:28:16.590909 21549 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b54963a844341f89e7c3fabd41452ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:28:16 UTC) from {username='kudu', principal='kudu/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:42102
I20260925 07:28:16.591008 21651 tablet_replica.cc:330] T 7b54963a844341f89e7c3fabd41452ae P 98d5e5d89d654dab96fc59e589ef6cfd: stopping tablet replica
I20260925 07:28:16.591099 21652 tablet_replica.cc:330] T 7b54963a844341f89e7c3fabd41452ae P 6d397540b57a4663935e8d3c4f5a417e: stopping tablet replica
I20260925 07:28:16.591154 21651 raft_consensus.cc:2229] T 7b54963a844341f89e7c3fabd41452ae P 98d5e5d89d654dab96fc59e589ef6cfd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 07:28:16.591181 21652 raft_consensus.cc:2229] T 7b54963a844341f89e7c3fabd41452ae P 6d397540b57a4663935e8d3c4f5a417e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 07:28:16.591272 21652 raft_consensus.cc:2258] T 7b54963a844341f89e7c3fabd41452ae P 6d397540b57a4663935e8d3c4f5a417e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:28:16.591272 21651 raft_consensus.cc:2258] T 7b54963a844341f89e7c3fabd41452ae P 98d5e5d89d654dab96fc59e589ef6cfd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:28:16.591593 21652 ts_tablet_manager.cc:1905] T 7b54963a844341f89e7c3fabd41452ae P 6d397540b57a4663935e8d3c4f5a417e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:28:16.591632 21651 ts_tablet_manager.cc:1905] T 7b54963a844341f89e7c3fabd41452ae P 98d5e5d89d654dab96fc59e589ef6cfd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:28:16.593195 21652 ts_tablet_manager.cc:1918] T 7b54963a844341f89e7c3fabd41452ae P 6d397540b57a4663935e8d3c4f5a417e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 07:28:16.593276 21652 log.cc:1196] T 7b54963a844341f89e7c3fabd41452ae P 6d397540b57a4663935e8d3c4f5a417e: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-2/wal/wals/7b54963a844341f89e7c3fabd41452ae
I20260925 07:28:16.593505 21652 ts_tablet_manager.cc:1939] T 7b54963a844341f89e7c3fabd41452ae P 6d397540b57a4663935e8d3c4f5a417e: Deleting consensus metadata
I20260925 07:28:16.593650 21651 ts_tablet_manager.cc:1918] T 7b54963a844341f89e7c3fabd41452ae P 98d5e5d89d654dab96fc59e589ef6cfd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 07:28:16.593726 21651 log.cc:1196] T 7b54963a844341f89e7c3fabd41452ae P 98d5e5d89d654dab96fc59e589ef6cfd: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-1/wal/wals/7b54963a844341f89e7c3fabd41452ae
I20260925 07:28:16.593894 21143 catalog_manager.cc:4841] TS 6d397540b57a4663935e8d3c4f5a417e (127.5.158.67:45323): tablet 7b54963a844341f89e7c3fabd41452ae (table test-table [id=98e9e0d383d64581852531f8450acfa8]) successfully deleted
I20260925 07:28:16.594043 21651 ts_tablet_manager.cc:1939] T 7b54963a844341f89e7c3fabd41452ae P 98d5e5d89d654dab96fc59e589ef6cfd: Deleting consensus metadata
I20260925 07:28:16.594559 21141 catalog_manager.cc:4841] TS 98d5e5d89d654dab96fc59e589ef6cfd (127.5.158.66:43065): tablet 7b54963a844341f89e7c3fabd41452ae (table test-table [id=98e9e0d383d64581852531f8450acfa8]) successfully deleted
W20260925 07:28:16.595626 21142 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.158.65:37045 (error 108)
I20260925 07:28:16.595800 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 21352
I20260925 07:28:16.597043 21625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:16.596892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:37045 (local address 127.0.0.1:33106)
0925 07:28:16.596962 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:16.597008 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.65:37045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260925 07:28:16.597167 21142 catalog_manager.cc:4584] TS 3d9933f641294109bd395addc5ab8c70 (127.5.158.65:37045): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7b54963a844341f89e7c3fabd41452ae: Network error: Client connection negotiation failed: client connection to 127.5.158.65:37045: connect: Connection refused (error 111)
I20260925 07:28:16.601028 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 21486
I20260925 07:28:16.605239 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 21124
I20260925 07:28:36.611557 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:32843
--webserver_interface=127.5.158.126
--webserver_port=44765
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:44845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.158.126:32843
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:28:36.684883 21654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:28:36.685070 21654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:28:36.685127 21654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:28:36.686175 21654 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260925 07:28:36.686211 21654 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260925 07:28:36.686224 21654 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260925 07:28:36.686236 21654 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:28:36.686249 21654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:28:36.686261 21654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:28:36.686278 21654 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:28:36.686290 21654 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:28:36.687597 21654 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:44845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.5.158.126:32843
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:32843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.158.126
--webserver_port=44765
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21654
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:28:36.687748 21654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:28:36.687901 21654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:28:36.696012 21661 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:28:36.696014 21660 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:28:36.696058 21654 server_base.cc:1034] running on GCE node
W20260925 07:28:36.696013 21663 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:28:36.696373 21654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:28:36.696643 21654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:28:36.697785 21654 hybrid_clock.cc:648] HybridClock initialized: now 1790321316697774 us; error 28 us; skew 500 ppm
Sep 25 07:28:36 dist-test-slave-1drq krb5kdc[21111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321316, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:28:36.705539 21654 init.cc:377] Logged in from keytab as kudu/127.5.158.126@KRBTEST.COM (short username kudu)
I20260925 07:28:36.705941 21654 webserver.cc:460] Webserver started at http://127.5.158.126:44765/ using document root <none> and password file <none>
I20260925 07:28:36.706128 21654 fs_manager.cc:359] Metadata directory not provided
I20260925 07:28:36.706175 21654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:28:36.707388 21654 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 07:28:36.707937 21670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:28:36.708073 21654 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 07:28:36.708169 21654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "94c99bc2c8fd47adb7b95b181d9246de"
format_stamp: "Formatted at 2026-09-25 07:28:15 on dist-test-slave-1drq"
server_key: "60aff4a17a032ff097c0baad88203862"
server_key_iv: "1d61bdd82e828a1a192ee19ad2b3e3d9"
server_key_version: "encryptionkey@0"
I20260925 07:28:36.708456 21654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:28:36.720116 21654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:28:36.721666 21654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:28:36.721788 21654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:28:36.725169 21654 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.126:32843
I20260925 07:28:36.725211 21722 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.126:32843 every 8 connection(s)
I20260925 07:28:36.725531 21654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:28:36.725955 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 21654
I20260925 07:28:36.726218 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.65:37045
--local_ip_for_outbound_sockets=127.5.158.65
--tserver_master_addrs=127.5.158.126:32843
--webserver_port=39241
--webserver_interface=127.5.158.65
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab
--principal=kudu/127.5.158.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:44845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 07:28:36.726989 21723 sys_catalog.cc:263] Verifying existing consensus state
I20260925 07:28:36.727566 21723 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de: Bootstrap starting.
I20260925 07:28:36.734918 21723 log.cc:824] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de: Log is configured to *not* fsync() on all Append() calls
I20260925 07:28:36.736573 21723 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 07:28:36.736852 21723 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de: Bootstrap complete.
I20260925 07:28:36.738809 21723 raft_consensus.cc:348] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94c99bc2c8fd47adb7b95b181d9246de" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 32843 } }
I20260925 07:28:36.738987 21723 raft_consensus.cc:729] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94c99bc2c8fd47adb7b95b181d9246de, State: Initialized, Role: FOLLOWER
I20260925 07:28:36.739121 21723 consensus_queue.cc:260] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94c99bc2c8fd47adb7b95b181d9246de" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 32843 } }
I20260925 07:28:36.739188 21723 raft_consensus.cc:388] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 07:28:36.739236 21723 raft_consensus.cc:482] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 07:28:36.739286 21723 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de [term 1 FOLLOWER]: Advancing to term 2
I20260925 07:28:36.740127 21723 raft_consensus.cc:504] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94c99bc2c8fd47adb7b95b181d9246de" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 32843 } }
I20260925 07:28:36.740228 21723 leader_election.cc:302] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 94c99bc2c8fd47adb7b95b181d9246de; no voters:
I20260925 07:28:36.740384 21723 leader_election.cc:288] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de [CANDIDATE]: Term 2 election: Requested vote from peers
I20260925 07:28:36.740418 21727 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de [term 2 FOLLOWER]: Leader election won for term 2
I20260925 07:28:36.740574 21727 raft_consensus.cc:686] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de [term 2 LEADER]: Becoming Leader. State: Replica: 94c99bc2c8fd47adb7b95b181d9246de, State: Running, Role: LEADER
I20260925 07:28:36.740643 21723 sys_catalog.cc:564] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:28:36.740710 21727 consensus_queue.cc:237] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94c99bc2c8fd47adb7b95b181d9246de" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 32843 } }
I20260925 07:28:36.740947 21727 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de [sys.catalog]: SysCatalogTable state changed. Reason: New leader 94c99bc2c8fd47adb7b95b181d9246de. Latest consensus state: current_term: 2 leader_uuid: "94c99bc2c8fd47adb7b95b181d9246de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94c99bc2c8fd47adb7b95b181d9246de" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 32843 } } }
I20260925 07:28:36.741034 21727 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de [sys.catalog]: This master's current role is: LEADER
I20260925 07:28:36.740953 21728 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "94c99bc2c8fd47adb7b95b181d9246de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94c99bc2c8fd47adb7b95b181d9246de" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 32843 } } }
I20260925 07:28:36.741210 21728 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de [sys.catalog]: This master's current role is: LEADER
I20260925 07:28:36.741362 21735 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 07:28:36.742018 21735 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 07:28:36.742110 21735 catalog_manager.cc:1259] Loaded cluster ID: c111839a6f3744caa17ed42fd6e29909
I20260925 07:28:36.742141 21735 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 07:28:36.743194 21735 catalog_manager.cc:1504] Loading token signing keys...
I20260925 07:28:36.743378 21735 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de: Loaded TSK: 0
I20260925 07:28:36.752930 21735 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de: Generated new TSK 1
I20260925 07:28:36.753000 21735 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 07:28:36.800659 21725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:28:36.800835 21725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:28:36.800870 21725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:28:36.802008 21725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:28:36.802062 21725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:28:36.802100 21725 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.65
I20260925 07:28:36.803444 21725 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:44845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab
--principal=kudu/127.5.158.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.65:37045
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.158.65
--webserver_port=39241
--tserver_master_addrs=127.5.158.126:32843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.65
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:28:36.803691 21725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:28:36.803890 21725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:28:36.808305 21751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:28:36.808316 21750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:28:36.808403 21753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:28:36.808399 21725 server_base.cc:1034] running on GCE node
I20260925 07:28:36.808679 21725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:28:36.808933 21725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:28:36.810101 21725 hybrid_clock.cc:648] HybridClock initialized: now 1790321316810082 us; error 31 us; skew 500 ppm
Sep 25 07:28:36 dist-test-slave-1drq krb5kdc[21111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321316, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:28:36.812625 21725 init.cc:377] Logged in from keytab as kudu/127.5.158.65@KRBTEST.COM (short username kudu)
I20260925 07:28:36.813030 21725 webserver.cc:460] Webserver started at http://127.5.158.65:39241/ using document root <none> and password file <none>
I20260925 07:28:36.813251 21725 fs_manager.cc:359] Metadata directory not provided
I20260925 07:28:36.813319 21725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:28:36.814512 21725 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260925 07:28:36.815133 21760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:28:36.815284 21725 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 07:28:36.815361 21725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-0/wal
uuid: "3d9933f641294109bd395addc5ab8c70"
format_stamp: "Formatted at 2026-09-25 07:28:16 on dist-test-slave-1drq"
server_key: "a35782025adaf52818d49771bb8a133c"
server_key_iv: "1fc684a67be1a898a1cc4aadc88a85e5"
server_key_version: "encryptionkey@0"
I20260925 07:28:36.815630 21725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:28:36.831437 21725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:28:36.833372 21725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:28:36.833518 21725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:28:36.833714 21725 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:28:36.834152 21767 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260925 07:28:36.835374 21725 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260925 07:28:36.835438 21725 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20260925 07:28:36.835464 21725 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260925 07:28:36.836040 21725 ts_tablet_manager.cc:610] Registered 1 tablets
I20260925 07:28:36.836082 21725 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260925 07:28:36.836124 21767 tablet_bootstrap.cc:492] T 7b54963a844341f89e7c3fabd41452ae P 3d9933f641294109bd395addc5ab8c70: Bootstrap starting.
I20260925 07:28:36.843055 21725 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.65:37045
I20260925 07:28:36.843173 21874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.65:37045 every 8 connection(s)
I20260925 07:28:36.843475 21725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
I20260925 07:28:36.846513 21767 log.cc:824] T 7b54963a844341f89e7c3fabd41452ae P 3d9933f641294109bd395addc5ab8c70: Log is configured to *not* fsync() on all Append() calls
I20260925 07:28:36.848502 21767 tablet_bootstrap.cc:492] T 7b54963a844341f89e7c3fabd41452ae P 3d9933f641294109bd395addc5ab8c70: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 07:28:36.848757 21767 tablet_bootstrap.cc:492] T 7b54963a844341f89e7c3fabd41452ae P 3d9933f641294109bd395addc5ab8c70: Bootstrap complete.
I20260925 07:28:36.849833 21767 ts_tablet_manager.cc:1397] T 7b54963a844341f89e7c3fabd41452ae P 3d9933f641294109bd395addc5ab8c70: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.012s
I20260925 07:28:36.850349 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 21725
I20260925 07:28:36.850593 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.66:43065
--local_ip_for_outbound_sockets=127.5.158.66
--tserver_master_addrs=127.5.158.126:32843
--webserver_port=45961
--webserver_interface=127.5.158.66
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab
--principal=kudu/127.5.158.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:44845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 07:28:36.851049 21767 raft_consensus.cc:348] T 7b54963a844341f89e7c3fabd41452ae P 3d9933f641294109bd395addc5ab8c70 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d9933f641294109bd395addc5ab8c70" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 37045 } } peers { permanent_uuid: "98d5e5d89d654dab96fc59e589ef6cfd" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 43065 } } peers { permanent_uuid: "6d397540b57a4663935e8d3c4f5a417e" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 45323 } }
I20260925 07:28:36.851317 21767 raft_consensus.cc:729] T 7b54963a844341f89e7c3fabd41452ae P 3d9933f641294109bd395addc5ab8c70 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d9933f641294109bd395addc5ab8c70, State: Initialized, Role: FOLLOWER
I20260925 07:28:36.851450 21767 consensus_queue.cc:260] T 7b54963a844341f89e7c3fabd41452ae P 3d9933f641294109bd395addc5ab8c70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d9933f641294109bd395addc5ab8c70" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 37045 } } peers { permanent_uuid: "98d5e5d89d654dab96fc59e589ef6cfd" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 43065 } } peers { permanent_uuid: "6d397540b57a4663935e8d3c4f5a417e" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 45323 } }
I20260925 07:28:36.851706 21767 ts_tablet_manager.cc:1428] T 7b54963a844341f89e7c3fabd41452ae P 3d9933f641294109bd395addc5ab8c70: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
Sep 25 07:28:36 dist-test-slave-1drq krb5kdc[21111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321316, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.65@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:28:36.856525 21877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:36.844149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.65:60851)
0925 07:28:36.844365 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:36.844398 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0925 07:28:36.844860 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:28:36.845635 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:28:36.845638 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:28:36.845954 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:28:36.846201 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:28:36.846207 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:28:36.847712 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:28:36.847713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:28:36.848860 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:28:36.848863 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:28:36.849653 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:28:36.849654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:28:36.849794 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:28:36.851895 (+ 2101us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:28:36.851911 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:28:36.853239 (+ 1328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:28:36.855011 (+ 1772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:28:36.855017 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:28:36.855019 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:28:36.855251 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:28:36.855472 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:28:36.855472 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:28:36.855473 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:28:36.855572 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:28:36.855944 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:28:36.855950 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:28:36.856081 (+ 131us) client_negotiation.cc:769] Sending connection context
0925 07:28:36.856182 (+ 101us) client_negotiation.cc:240] Negotiation successful
0925 07:28:36.856345 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":78,"threads_started":1}
I20260925 07:28:36.856550 21878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:36.844236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:60851 (local address 127.5.158.126:32843)
0925 07:28:36.845215 (+ 979us) server_negotiation.cc:206] Beginning negotiation
0925 07:28:36.845225 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 07:28:36.845267 (+ 42us) server_negotiation.cc:407] Connection header received
0925 07:28:36.845392 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:28:36.845397 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:28:36.845458 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:28:36.845602 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:28:36.846329 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:28:36.847594 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:28:36.848949 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:28:36.849582 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:28:36.849669 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:28:36.853334 (+ 3665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:28:36.853338 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:28:36.853338 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:28:36.853349 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:28:36.854916 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:28:36.855323 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:28:36.855326 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:28:36.855326 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:28:36.855407 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:28:36.855658 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:28:36.855659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:28:36.855659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:28:36.855858 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:28:36.855970 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0925 07:28:36.856233 (+ 263us) server_negotiation.cc:299] Negotiation successful
0925 07:28:36.856345 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":930,"thread_start_us":79,"threads_started":1}
I20260925 07:28:36.857405 21875 heartbeater.cc:347] Connected to a master server at 127.5.158.126:32843
I20260925 07:28:36.857489 21875 heartbeater.cc:464] Registering TS with master...
I20260925 07:28:36.857642 21875 heartbeater.cc:511] Master 127.5.158.126:32843 requested a full tablet report, sending...
I20260925 07:28:36.858347 21687 ts_manager.cc:194] Registered new tserver with Master: 3d9933f641294109bd395addc5ab8c70 (127.5.158.65:37045)
I20260925 07:28:36.859412 21687 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.65@KRBTEST.COM'} at 127.5.158.65:60851
I20260925 07:28:36.860270 21875 heartbeater.cc:503] Master 127.5.158.126:32843 was elected leader, sending a full tablet report...
Sep 25 07:28:36 dist-test-slave-1drq krb5kdc[21111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321316, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.126@KRBTEST.COM for kudu/127.5.158.65@KRBTEST.COM
I20260925 07:28:36.865998 21882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:36.858862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:37045 (local address 127.0.0.1:56198)
0925 07:28:36.859206 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:36.859234 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0925 07:28:36.859285 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:28:36.859624 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:28:36.859628 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:28:36.859910 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:28:36.859945 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:28:36.859947 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:28:36.860734 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:28:36.860735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:28:36.861338 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:28:36.861340 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:28:36.861741 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:28:36.861741 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:28:36.861805 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:28:36.862226 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:28:36.862231 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:28:36.863382 (+ 1151us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:28:36.864863 (+ 1481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:28:36.864869 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:28:36.864871 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:28:36.865036 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:28:36.865331 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:28:36.865332 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:28:36.865332 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:28:36.865394 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:28:36.865649 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:28:36.865655 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:28:36.865703 (+ 48us) client_negotiation.cc:769] Sending connection context
0925 07:28:36.865796 (+ 93us) client_negotiation.cc:240] Negotiation successful
0925 07:28:36.865905 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":72,"threads_started":1}
I20260925 07:28:36.866009 21883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:36.859075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56198 (local address 127.5.158.65:37045)
0925 07:28:36.859379 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0925 07:28:36.859382 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 07:28:36.859390 (+ 8us) server_negotiation.cc:407] Connection header received
0925 07:28:36.859415 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:28:36.859416 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:28:36.859429 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:28:36.859570 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:28:36.860032 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:28:36.860658 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:28:36.861415 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:28:36.861671 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:28:36.861741 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:28:36.863458 (+ 1717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:28:36.863466 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:28:36.863466 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:28:36.863472 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:28:36.864795 (+ 1323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:28:36.865186 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:28:36.865189 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:28:36.865189 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:28:36.865256 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:28:36.865452 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:28:36.865453 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:28:36.865453 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:28:36.865582 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:28:36.865666 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0925 07:28:36.865832 (+ 166us) server_negotiation.cc:299] Negotiation successful
0925 07:28:36.865920 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":57,"threads_started":1}
I20260925 07:28:36.866621 21809 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b54963a844341f89e7c3fabd41452ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:28:16 UTC) from {username='kudu', principal='kudu/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:56198
I20260925 07:28:36.867043 21884 tablet_replica.cc:330] stopping tablet replica
I20260925 07:28:36.867161 21884 raft_consensus.cc:2229] T 7b54963a844341f89e7c3fabd41452ae P 3d9933f641294109bd395addc5ab8c70 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 07:28:36.867219 21884 raft_consensus.cc:2258] T 7b54963a844341f89e7c3fabd41452ae P 3d9933f641294109bd395addc5ab8c70 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:28:36.867478 21884 ts_tablet_manager.cc:1905] T 7b54963a844341f89e7c3fabd41452ae P 3d9933f641294109bd395addc5ab8c70: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:28:36.869529 21884 ts_tablet_manager.cc:1918] T 7b54963a844341f89e7c3fabd41452ae P 3d9933f641294109bd395addc5ab8c70: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 07:28:36.869601 21884 log.cc:1196] T 7b54963a844341f89e7c3fabd41452ae P 3d9933f641294109bd395addc5ab8c70: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-0/wal/wals/7b54963a844341f89e7c3fabd41452ae
I20260925 07:28:36.869791 21884 ts_tablet_manager.cc:1939] T 7b54963a844341f89e7c3fabd41452ae P 3d9933f641294109bd395addc5ab8c70: Deleting consensus metadata
I20260925 07:28:36.870193 21673 catalog_manager.cc:4841] TS 3d9933f641294109bd395addc5ab8c70 (127.5.158.65:37045): tablet 7b54963a844341f89e7c3fabd41452ae (table test-table [id=98e9e0d383d64581852531f8450acfa8]) successfully deleted
W20260925 07:28:36.930727 21881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:28:36.930895 21881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:28:36.930915 21881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:28:36.931887 21881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:28:36.931922 21881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:28:36.931963 21881 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.66
I20260925 07:28:36.933207 21881 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:44845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab
--principal=kudu/127.5.158.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.66:43065
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.158.66
--webserver_port=45961
--tserver_master_addrs=127.5.158.126:32843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21881
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.66
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:28:36.933398 21881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:28:36.933573 21881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:28:36.938191 21889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:28:36.938241 21890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:28:36.938362 21881 server_base.cc:1034] running on GCE node
W20260925 07:28:36.938450 21892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:28:36.938628 21881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:28:36.938791 21881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:28:36.939914 21881 hybrid_clock.cc:648] HybridClock initialized: now 1790321316939908 us; error 23 us; skew 500 ppm
Sep 25 07:28:36 dist-test-slave-1drq krb5kdc[21111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321316, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:28:36.942157 21881 init.cc:377] Logged in from keytab as kudu/127.5.158.66@KRBTEST.COM (short username kudu)
I20260925 07:28:36.942526 21881 webserver.cc:460] Webserver started at http://127.5.158.66:45961/ using document root <none> and password file <none>
I20260925 07:28:36.942725 21881 fs_manager.cc:359] Metadata directory not provided
I20260925 07:28:36.942790 21881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:28:36.943859 21881 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:28:36.944275 21899 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:28:36.944425 21881 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 07:28:36.944487 21881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-1/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-1/wal
uuid: "98d5e5d89d654dab96fc59e589ef6cfd"
format_stamp: "Formatted at 2026-09-25 07:28:16 on dist-test-slave-1drq"
server_key: "ed675f764ddbc1fc0df3b050bb9dfc3a"
server_key_iv: "38b84b7c338f9ff97f9466b778bd7b49"
server_key_version: "encryptionkey@0"
I20260925 07:28:36.944733 21881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:28:36.973359 21881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:28:36.974759 21881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:28:36.974848 21881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:28:36.974988 21881 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:28:36.975205 21881 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:28:36.975230 21881 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:28:36.975246 21881 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:28:36.975257 21881 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:28:36.980537 21881 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.66:43065
I20260925 07:28:36.980567 22012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.66:43065 every 8 connection(s)
I20260925 07:28:36.980835 21881 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
Sep 25 07:28:36 dist-test-slave-1drq krb5kdc[21111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321316, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.66@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:28:36.986125 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 21881
I20260925 07:28:36.986377 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.67:45323
--local_ip_for_outbound_sockets=127.5.158.67
--tserver_master_addrs=127.5.158.126:32843
--webserver_port=38377
--webserver_interface=127.5.158.67
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab
--principal=kudu/127.5.158.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:44845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 07:28:36.988552 21878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:36.981405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:55789 (local address 127.5.158.126:32843)
0925 07:28:36.981439 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0925 07:28:36.981441 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:28:36.981607 (+ 166us) server_negotiation.cc:407] Connection header received
0925 07:28:36.981943 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:28:36.981943 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:28:36.981958 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:28:36.982000 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:28:36.982714 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:28:36.983147 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:28:36.983944 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:28:36.984116 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:28:36.984158 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:28:36.985839 (+ 1681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:28:36.985840 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:28:36.985841 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:28:36.985847 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:28:36.987375 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:28:36.987731 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:28:36.987732 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:28:36.987732 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:28:36.987786 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:28:36.988001 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:28:36.988002 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:28:36.988002 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:28:36.988100 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:28:36.988165 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 07:28:36.988390 (+ 225us) server_negotiation.cc:299] Negotiation successful
0925 07:28:36.988471 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260925 07:28:36.988680 22015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:36.981405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.66:55789)
0925 07:28:36.981519 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:36.981545 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0925 07:28:36.981875 (+ 330us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:28:36.982041 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:28:36.982043 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:28:36.982505 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:28:36.982653 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:28:36.982656 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:28:36.983211 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:28:36.983212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:28:36.983884 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:28:36.983885 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:28:36.984173 (+ 288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:28:36.984174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:28:36.984254 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:28:36.984586 (+ 332us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:28:36.984593 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:28:36.985769 (+ 1176us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:28:36.987455 (+ 1686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:28:36.987461 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:28:36.987463 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:28:36.987651 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:28:36.987843 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:28:36.987843 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:28:36.987844 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:28:36.987942 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:28:36.988158 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:28:36.988163 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:28:36.988281 (+ 118us) client_negotiation.cc:769] Sending connection context
0925 07:28:36.988372 (+ 91us) client_negotiation.cc:240] Negotiation successful
0925 07:28:36.988496 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":68,"threads_started":1}
I20260925 07:28:36.989063 22013 heartbeater.cc:347] Connected to a master server at 127.5.158.126:32843
I20260925 07:28:36.989157 22013 heartbeater.cc:464] Registering TS with master...
I20260925 07:28:36.989320 22013 heartbeater.cc:511] Master 127.5.158.126:32843 requested a full tablet report, sending...
I20260925 07:28:36.989615 21687 ts_manager.cc:194] Registered new tserver with Master: 98d5e5d89d654dab96fc59e589ef6cfd (127.5.158.66:43065)
I20260925 07:28:36.989946 21687 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.66@KRBTEST.COM'} at 127.5.158.66:55789
W20260925 07:28:37.059751 22016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:28:37.059903 22016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:28:37.059923 22016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:28:37.060959 22016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:28:37.060995 22016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:28:37.061039 22016 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.67
I20260925 07:28:37.062280 22016 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:44845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab
--principal=kudu/127.5.158.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.67:45323
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.158.67
--webserver_port=38377
--tserver_master_addrs=127.5.158.126:32843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22016
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.67
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:28:37.062475 22016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:28:37.062655 22016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 07:28:37.066987 22016 server_base.cc:1034] running on GCE node
W20260925 07:28:37.067046 22021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:28:37.066996 22022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:28:37.066980 22024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:28:37.067420 22016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:28:37.067677 22016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:28:37.068837 22016 hybrid_clock.cc:648] HybridClock initialized: now 1790321317068827 us; error 21 us; skew 500 ppm
Sep 25 07:28:37 dist-test-slave-1drq krb5kdc[21111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321317, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:28:37.070834 22016 init.cc:377] Logged in from keytab as kudu/127.5.158.67@KRBTEST.COM (short username kudu)
I20260925 07:28:37.071208 22016 webserver.cc:460] Webserver started at http://127.5.158.67:38377/ using document root <none> and password file <none>
I20260925 07:28:37.071404 22016 fs_manager.cc:359] Metadata directory not provided
I20260925 07:28:37.071471 22016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:28:37.072536 22016 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 07:28:37.072973 22031 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:28:37.073143 22016 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 07:28:37.073218 22016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-2/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-2/wal
uuid: "6d397540b57a4663935e8d3c4f5a417e"
format_stamp: "Formatted at 2026-09-25 07:28:16 on dist-test-slave-1drq"
server_key: "38f38507c7e2c77c84af8fd08ed34aa4"
server_key_iv: "2704311d537a11eb8df9d149c48cce82"
server_key_version: "encryptionkey@0"
I20260925 07:28:37.073477 22016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:28:37.089468 22016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:28:37.091090 22016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:28:37.091228 22016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:28:37.091496 22016 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:28:37.091804 22016 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:28:37.091866 22016 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:28:37.091914 22016 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:28:37.091941 22016 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:28:37.097446 22016 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.67:45323
I20260925 07:28:37.097491 22144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.67:45323 every 8 connection(s)
I20260925 07:28:37.097748 22016 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
I20260925 07:28:37.100725 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 22016
Sep 25 07:28:37 dist-test-slave-1drq krb5kdc[21111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321317, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.67@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:28:37.106389 21878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:37.098249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:38193 (local address 127.5.158.126:32843)
0925 07:28:37.098320 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0925 07:28:37.098322 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:28:37.098476 (+ 154us) server_negotiation.cc:407] Connection header received
0925 07:28:37.098823 (+ 347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:28:37.098824 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:28:37.098833 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:28:37.098877 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:28:37.099638 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:28:37.100078 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:28:37.101146 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:28:37.101441 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:28:37.101498 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:28:37.103556 (+ 2058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:28:37.103557 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:28:37.103557 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:28:37.103563 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:28:37.105045 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:28:37.105479 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:28:37.105480 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:28:37.105481 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:28:37.105552 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:28:37.105773 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:28:37.105774 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:28:37.105774 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:28:37.105874 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:28:37.105958 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0925 07:28:37.106209 (+ 251us) server_negotiation.cc:299] Negotiation successful
0925 07:28:37.106301 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260925 07:28:37.106521 22147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:37.098243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.67:38193)
0925 07:28:37.098378 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:37.098406 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0925 07:28:37.098747 (+ 341us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:28:37.098917 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:28:37.098920 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:28:37.099404 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:28:37.099571 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:28:37.099574 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:28:37.100136 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:28:37.100137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:28:37.101036 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:28:37.101039 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:28:37.101514 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:28:37.101515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:28:37.101605 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:28:37.102031 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:28:37.102040 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:28:37.103456 (+ 1416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:28:37.105192 (+ 1736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:28:37.105198 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:28:37.105199 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:28:37.105412 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:28:37.105610 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:28:37.105611 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:28:37.105611 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:28:37.105705 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:28:37.105949 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:28:37.105955 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:28:37.106080 (+ 125us) client_negotiation.cc:769] Sending connection context
0925 07:28:37.106173 (+ 93us) client_negotiation.cc:240] Negotiation successful
0925 07:28:37.106319 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":75,"threads_started":1}
I20260925 07:28:37.106895 22145 heartbeater.cc:347] Connected to a master server at 127.5.158.126:32843
I20260925 07:28:37.107003 22145 heartbeater.cc:464] Registering TS with master...
I20260925 07:28:37.107194 22145 heartbeater.cc:511] Master 127.5.158.126:32843 requested a full tablet report, sending...
I20260925 07:28:37.107542 21687 ts_manager.cc:194] Registered new tserver with Master: 6d397540b57a4663935e8d3c4f5a417e (127.5.158.67:45323)
I20260925 07:28:37.107700 22148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:37.101070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45504 (local address 127.5.158.126:32843)
0925 07:28:37.101199 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0925 07:28:37.101201 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:28:37.101302 (+ 101us) server_negotiation.cc:407] Connection header received
0925 07:28:37.101346 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:28:37.101346 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:28:37.101354 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:28:37.101397 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:28:37.102056 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:28:37.102689 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:28:37.103669 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:28:37.103930 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:28:37.103995 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:28:37.105161 (+ 1166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:28:37.105162 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:28:37.105162 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:28:37.105184 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:28:37.106586 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:28:37.106937 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:28:37.106938 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:28:37.106939 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:28:37.106996 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:28:37.107219 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:28:37.107220 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:28:37.107220 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:28:37.107351 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:28:37.107497 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0925 07:28:37.107565 (+ 68us) server_negotiation.cc:299] Negotiation successful
0925 07:28:37.107631 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":69,"threads_started":1}
I20260925 07:28:37.107924 21687 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.67@KRBTEST.COM'} at 127.5.158.67:38193
I20260925 07:28:37.108251 5753 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 07:28:37.110193 22148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:37.108568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45514 (local address 127.5.158.126:32843)
0925 07:28:37.108604 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0925 07:28:37.108606 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:28:37.108639 (+ 33us) server_negotiation.cc:407] Connection header received
0925 07:28:37.108688 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:28:37.108689 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:28:37.108711 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:28:37.108755 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 07:28:37.108813 (+ 58us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:28:37.109220 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:28:37.109779 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:28:37.109924 (+ 145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:28:37.109966 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:28:37.110043 (+ 77us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 07:28:37.110082 (+ 39us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0925 07:28:37.110161 (+ 79us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45514: token expired
Metrics: {"server-negotiator.queue_time_us":21}
W20260925 07:28:37.110242 22148 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45514: token expired
W20260925 07:28:37.110692 21620 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.5.158.126:32843)
I20260925 07:28:37.118312 21878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:37.111555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45516 (local address 127.5.158.126:32843)
0925 07:28:37.111598 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 07:28:37.111599 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:28:37.111636 (+ 37us) server_negotiation.cc:407] Connection header received
0925 07:28:37.111759 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:28:37.111760 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:28:37.111770 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:28:37.111824 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:28:37.112406 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:28:37.112747 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:28:37.113810 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:28:37.113986 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:28:37.114040 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:28:37.115742 (+ 1702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:28:37.115743 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:28:37.115744 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:28:37.115749 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:28:37.117335 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:28:37.117667 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:28:37.117668 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:28:37.117668 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:28:37.117713 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:28:37.117889 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:28:37.117890 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:28:37.117890 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:28:37.117986 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:28:37.118060 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 07:28:37.118192 (+ 132us) server_negotiation.cc:299] Negotiation successful
0925 07:28:37.118250 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 07:28:37.119318 21687 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45516:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 07:28:37.120754 21687 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 07:28:37.122491 21809 tablet_service.cc:1467] Processing CreateTablet for tablet 59c05f9a574a4b2db0631fc455feedfe (DEFAULT_TABLE table=test-table [id=1691a5094a1a4a8b8cd2c775f65839bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:28:37.122656 21809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59c05f9a574a4b2db0631fc455feedfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:28:37.123833 21767 tablet_bootstrap.cc:492] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70: Bootstrap starting.
I20260925 07:28:37.124112 21767 tablet_bootstrap.cc:654] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70: Neither blocks nor log segments found. Creating new log.
I20260925 07:28:37.124640 21767 tablet_bootstrap.cc:492] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70: No bootstrap required, opened a new log
I20260925 07:28:37.124675 21767 ts_tablet_manager.cc:1397] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260925 07:28:37.124945 21767 raft_consensus.cc:348] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98d5e5d89d654dab96fc59e589ef6cfd" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 43065 } } peers { permanent_uuid: "6d397540b57a4663935e8d3c4f5a417e" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 45323 } } peers { permanent_uuid: "3d9933f641294109bd395addc5ab8c70" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 37045 } }
I20260925 07:28:37.124997 21767 raft_consensus.cc:374] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:28:37.125033 21767 raft_consensus.cc:729] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d9933f641294109bd395addc5ab8c70, State: Initialized, Role: FOLLOWER
I20260925 07:28:37.125120 21767 consensus_queue.cc:260] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98d5e5d89d654dab96fc59e589ef6cfd" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 43065 } } peers { permanent_uuid: "6d397540b57a4663935e8d3c4f5a417e" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 45323 } } peers { permanent_uuid: "3d9933f641294109bd395addc5ab8c70" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 37045 } }
I20260925 07:28:37.125222 21767 ts_tablet_manager.cc:1428] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260925 07:28:37.125845 21882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:37.122293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.67:45323 (local address 127.0.0.1:52646)
0925 07:28:37.122377 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:37.122386 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 07:28:37.122418 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:28:37.122758 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:28:37.122758 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:28:37.122760 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:28:37.122803 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:28:37.122804 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:28:37.123881 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:28:37.123882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:28:37.124773 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:28:37.124786 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:28:37.125621 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:28:37.125622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:28:37.125682 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:28:37.125699 (+ 17us) client_negotiation.cc:769] Sending connection context
0925 07:28:37.125750 (+ 51us) client_negotiation.cc:240] Negotiation successful
0925 07:28:37.125787 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70,"mutex_wait_us":32}
I20260925 07:28:37.126012 22151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:37.122300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.66:43065 (local address 127.0.0.1:47060)
0925 07:28:37.122403 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:37.122412 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 07:28:37.122454 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:28:37.122795 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:28:37.122796 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:28:37.122797 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:28:37.122825 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:28:37.122826 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:28:37.124011 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:28:37.124012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:28:37.125002 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:28:37.125005 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:28:37.125787 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:28:37.125788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:28:37.125828 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:28:37.125830 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 07:28:37.125864 (+ 34us) client_negotiation.cc:240] Negotiation successful
0925 07:28:37.125905 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"mutex_wait_us":2,"thread_start_us":64,"threads_started":1}
I20260925 07:28:37.126082 22153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:37.122486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47060 (local address 127.5.158.66:43065)
0925 07:28:37.122660 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0925 07:28:37.122663 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 07:28:37.122669 (+ 6us) server_negotiation.cc:407] Connection header received
0925 07:28:37.122702 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:28:37.122704 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:28:37.122717 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:28:37.122889 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:28:37.122971 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:28:37.123944 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:28:37.125079 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:28:37.125716 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:28:37.125786 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:28:37.125833 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 07:28:37.125884 (+ 51us) server_negotiation.cc:299] Negotiation successful
0925 07:28:37.126005 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":81,"threads_started":1}
I20260925 07:28:37.126184 22152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:37.122478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52646 (local address 127.5.158.67:45323)
0925 07:28:37.122602 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0925 07:28:37.122605 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 07:28:37.122614 (+ 9us) server_negotiation.cc:407] Connection header received
0925 07:28:37.122644 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:28:37.122645 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:28:37.122658 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:28:37.122867 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:28:37.122931 (+ 64us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:28:37.123805 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:28:37.124862 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:28:37.125493 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:28:37.125988 (+ 495us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:28:37.126043 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 07:28:37.126066 (+ 23us) server_negotiation.cc:299] Negotiation successful
0925 07:28:37.126107 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":71,"threads_started":1}
I20260925 07:28:37.126585 21947 tablet_service.cc:1467] Processing CreateTablet for tablet 59c05f9a574a4b2db0631fc455feedfe (DEFAULT_TABLE table=test-table [id=1691a5094a1a4a8b8cd2c775f65839bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:28:37.126816 21947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59c05f9a574a4b2db0631fc455feedfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:28:37.126752 22079 tablet_service.cc:1467] Processing CreateTablet for tablet 59c05f9a574a4b2db0631fc455feedfe (DEFAULT_TABLE table=test-table [id=1691a5094a1a4a8b8cd2c775f65839bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:28:37.127031 22079 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59c05f9a574a4b2db0631fc455feedfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:28:37.129051 22155 tablet_bootstrap.cc:492] T 59c05f9a574a4b2db0631fc455feedfe P 98d5e5d89d654dab96fc59e589ef6cfd: Bootstrap starting.
I20260925 07:28:37.129340 22156 tablet_bootstrap.cc:492] T 59c05f9a574a4b2db0631fc455feedfe P 6d397540b57a4663935e8d3c4f5a417e: Bootstrap starting.
I20260925 07:28:37.129601 22155 tablet_bootstrap.cc:654] T 59c05f9a574a4b2db0631fc455feedfe P 98d5e5d89d654dab96fc59e589ef6cfd: Neither blocks nor log segments found. Creating new log.
I20260925 07:28:37.129804 22156 tablet_bootstrap.cc:654] T 59c05f9a574a4b2db0631fc455feedfe P 6d397540b57a4663935e8d3c4f5a417e: Neither blocks nor log segments found. Creating new log.
I20260925 07:28:37.130046 22155 log.cc:824] T 59c05f9a574a4b2db0631fc455feedfe P 98d5e5d89d654dab96fc59e589ef6cfd: Log is configured to *not* fsync() on all Append() calls
I20260925 07:28:37.130263 22156 log.cc:824] T 59c05f9a574a4b2db0631fc455feedfe P 6d397540b57a4663935e8d3c4f5a417e: Log is configured to *not* fsync() on all Append() calls
I20260925 07:28:37.130693 22155 tablet_bootstrap.cc:492] T 59c05f9a574a4b2db0631fc455feedfe P 98d5e5d89d654dab96fc59e589ef6cfd: No bootstrap required, opened a new log
I20260925 07:28:37.130755 22155 ts_tablet_manager.cc:1397] T 59c05f9a574a4b2db0631fc455feedfe P 98d5e5d89d654dab96fc59e589ef6cfd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 07:28:37.130867 22156 tablet_bootstrap.cc:492] T 59c05f9a574a4b2db0631fc455feedfe P 6d397540b57a4663935e8d3c4f5a417e: No bootstrap required, opened a new log
I20260925 07:28:37.130920 22156 ts_tablet_manager.cc:1397] T 59c05f9a574a4b2db0631fc455feedfe P 6d397540b57a4663935e8d3c4f5a417e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 07:28:37.131915 22155 raft_consensus.cc:348] T 59c05f9a574a4b2db0631fc455feedfe P 98d5e5d89d654dab96fc59e589ef6cfd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98d5e5d89d654dab96fc59e589ef6cfd" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 43065 } } peers { permanent_uuid: "6d397540b57a4663935e8d3c4f5a417e" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 45323 } } peers { permanent_uuid: "3d9933f641294109bd395addc5ab8c70" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 37045 } }
I20260925 07:28:37.132043 22155 raft_consensus.cc:374] T 59c05f9a574a4b2db0631fc455feedfe P 98d5e5d89d654dab96fc59e589ef6cfd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:28:37.132082 22155 raft_consensus.cc:729] T 59c05f9a574a4b2db0631fc455feedfe P 98d5e5d89d654dab96fc59e589ef6cfd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98d5e5d89d654dab96fc59e589ef6cfd, State: Initialized, Role: FOLLOWER
I20260925 07:28:37.131994 22156 raft_consensus.cc:348] T 59c05f9a574a4b2db0631fc455feedfe P 6d397540b57a4663935e8d3c4f5a417e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98d5e5d89d654dab96fc59e589ef6cfd" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 43065 } } peers { permanent_uuid: "6d397540b57a4663935e8d3c4f5a417e" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 45323 } } peers { permanent_uuid: "3d9933f641294109bd395addc5ab8c70" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 37045 } }
I20260925 07:28:37.132121 22156 raft_consensus.cc:374] T 59c05f9a574a4b2db0631fc455feedfe P 6d397540b57a4663935e8d3c4f5a417e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:28:37.132157 22156 raft_consensus.cc:729] T 59c05f9a574a4b2db0631fc455feedfe P 6d397540b57a4663935e8d3c4f5a417e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d397540b57a4663935e8d3c4f5a417e, State: Initialized, Role: FOLLOWER
I20260925 07:28:37.132176 22155 consensus_queue.cc:260] T 59c05f9a574a4b2db0631fc455feedfe P 98d5e5d89d654dab96fc59e589ef6cfd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98d5e5d89d654dab96fc59e589ef6cfd" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 43065 } } peers { permanent_uuid: "6d397540b57a4663935e8d3c4f5a417e" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 45323 } } peers { permanent_uuid: "3d9933f641294109bd395addc5ab8c70" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 37045 } }
I20260925 07:28:37.132243 22156 consensus_queue.cc:260] T 59c05f9a574a4b2db0631fc455feedfe P 6d397540b57a4663935e8d3c4f5a417e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98d5e5d89d654dab96fc59e589ef6cfd" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 43065 } } peers { permanent_uuid: "6d397540b57a4663935e8d3c4f5a417e" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 45323 } } peers { permanent_uuid: "3d9933f641294109bd395addc5ab8c70" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 37045 } }
I20260925 07:28:37.132404 22155 ts_tablet_manager.cc:1428] T 59c05f9a574a4b2db0631fc455feedfe P 98d5e5d89d654dab96fc59e589ef6cfd: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 07:28:37.132426 22013 heartbeater.cc:503] Master 127.5.158.126:32843 was elected leader, sending a full tablet report...
I20260925 07:28:37.132459 22145 heartbeater.cc:503] Master 127.5.158.126:32843 was elected leader, sending a full tablet report...
I20260925 07:28:37.132460 22156 ts_tablet_manager.cc:1428] T 59c05f9a574a4b2db0631fc455feedfe P 6d397540b57a4663935e8d3c4f5a417e: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260925 07:28:37.173942 21880 raft_consensus.cc:482] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 07:28:37.174017 21880 raft_consensus.cc:504] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98d5e5d89d654dab96fc59e589ef6cfd" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 43065 } } peers { permanent_uuid: "6d397540b57a4663935e8d3c4f5a417e" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 45323 } } peers { permanent_uuid: "3d9933f641294109bd395addc5ab8c70" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 37045 } }
I20260925 07:28:37.174279 21880 leader_election.cc:288] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98d5e5d89d654dab96fc59e589ef6cfd (127.5.158.66:43065), 6d397540b57a4663935e8d3c4f5a417e (127.5.158.67:45323)
I20260925 07:28:37.176849 22152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:37.174466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:46265 (local address 127.5.158.67:45323)
0925 07:28:37.174504 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 07:28:37.174505 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:28:37.174592 (+ 87us) server_negotiation.cc:407] Connection header received
0925 07:28:37.174639 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:28:37.174640 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:28:37.174647 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:28:37.174696 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:28:37.174818 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:28:37.175175 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:28:37.176059 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:28:37.176517 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:28:37.176570 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:28:37.176575 (+ 5us) server_negotiation.cc:1035] Waiting for connection context
0925 07:28:37.176756 (+ 181us) server_negotiation.cc:299] Negotiation successful
0925 07:28:37.176791 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 07:28:37.176879 22153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:37.174468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:35673 (local address 127.5.158.66:43065)
0925 07:28:37.174501 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0925 07:28:37.174503 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:28:37.174511 (+ 8us) server_negotiation.cc:407] Connection header received
0925 07:28:37.174531 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:28:37.174532 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:28:37.174538 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:28:37.174596 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:28:37.174707 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:28:37.175136 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:28:37.176111 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:28:37.176532 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:28:37.176614 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:28:37.176621 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 07:28:37.176790 (+ 169us) server_negotiation.cc:299] Negotiation successful
0925 07:28:37.176821 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 07:28:37.176885 21877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:37.174371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.66:43065 (local address 127.5.158.65:35673)
0925 07:28:37.174404 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:37.174412 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 07:28:37.174454 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:28:37.174606 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:28:37.174607 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:28:37.174609 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:28:37.174652 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:28:37.174654 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:28:37.175199 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:28:37.175199 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:28:37.176009 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:28:37.176012 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:28:37.176621 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:28:37.176622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:28:37.176701 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:28:37.176704 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 07:28:37.176770 (+ 66us) client_negotiation.cc:240] Negotiation successful
0925 07:28:37.176805 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17,"mutex_wait_us":35}
I20260925 07:28:37.176894 22161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:37.174401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.67:45323 (local address 127.5.158.65:46265)
0925 07:28:37.174536 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:37.174545 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 07:28:37.174589 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:28:37.174705 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:28:37.174705 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:28:37.174706 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:28:37.174766 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:28:37.174767 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:28:37.175250 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:28:37.175251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:28:37.175967 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:28:37.175969 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:28:37.176606 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:28:37.176607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:28:37.176680 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:28:37.176694 (+ 14us) client_negotiation.cc:769] Sending connection context
0925 07:28:37.176737 (+ 43us) client_negotiation.cc:240] Negotiation successful
0925 07:28:37.176793 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":1,"thread_start_us":72,"threads_started":1}
I20260925 07:28:37.177279 22099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59c05f9a574a4b2db0631fc455feedfe" candidate_uuid: "3d9933f641294109bd395addc5ab8c70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d397540b57a4663935e8d3c4f5a417e" is_pre_election: true
I20260925 07:28:37.177299 21967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59c05f9a574a4b2db0631fc455feedfe" candidate_uuid: "3d9933f641294109bd395addc5ab8c70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98d5e5d89d654dab96fc59e589ef6cfd" is_pre_election: true
I20260925 07:28:37.177384 22099 raft_consensus.cc:2454] T 59c05f9a574a4b2db0631fc455feedfe P 6d397540b57a4663935e8d3c4f5a417e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d9933f641294109bd395addc5ab8c70 in term 0.
I20260925 07:28:37.177392 21967 raft_consensus.cc:2454] T 59c05f9a574a4b2db0631fc455feedfe P 98d5e5d89d654dab96fc59e589ef6cfd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d9933f641294109bd395addc5ab8c70 in term 0.
I20260925 07:28:37.177549 21764 leader_election.cc:302] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d9933f641294109bd395addc5ab8c70, 6d397540b57a4663935e8d3c4f5a417e; no voters:
I20260925 07:28:37.177646 21880 raft_consensus.cc:2781] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 07:28:37.177681 21880 raft_consensus.cc:482] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 07:28:37.177700 21880 raft_consensus.cc:3037] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:28:37.178216 21880 raft_consensus.cc:504] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98d5e5d89d654dab96fc59e589ef6cfd" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 43065 } } peers { permanent_uuid: "6d397540b57a4663935e8d3c4f5a417e" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 45323 } } peers { permanent_uuid: "3d9933f641294109bd395addc5ab8c70" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 37045 } }
I20260925 07:28:37.178330 21880 leader_election.cc:288] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70 [CANDIDATE]: Term 1 election: Requested vote from peers 98d5e5d89d654dab96fc59e589ef6cfd (127.5.158.66:43065), 6d397540b57a4663935e8d3c4f5a417e (127.5.158.67:45323)
I20260925 07:28:37.178444 22099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59c05f9a574a4b2db0631fc455feedfe" candidate_uuid: "3d9933f641294109bd395addc5ab8c70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d397540b57a4663935e8d3c4f5a417e"
I20260925 07:28:37.178448 21967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59c05f9a574a4b2db0631fc455feedfe" candidate_uuid: "3d9933f641294109bd395addc5ab8c70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98d5e5d89d654dab96fc59e589ef6cfd"
I20260925 07:28:37.178517 22099 raft_consensus.cc:3037] T 59c05f9a574a4b2db0631fc455feedfe P 6d397540b57a4663935e8d3c4f5a417e [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:28:37.178516 21967 raft_consensus.cc:3037] T 59c05f9a574a4b2db0631fc455feedfe P 98d5e5d89d654dab96fc59e589ef6cfd [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:28:37.179023 21967 raft_consensus.cc:2454] T 59c05f9a574a4b2db0631fc455feedfe P 98d5e5d89d654dab96fc59e589ef6cfd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d9933f641294109bd395addc5ab8c70 in term 1.
I20260925 07:28:37.179021 22099 raft_consensus.cc:2454] T 59c05f9a574a4b2db0631fc455feedfe P 6d397540b57a4663935e8d3c4f5a417e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d9933f641294109bd395addc5ab8c70 in term 1.
I20260925 07:28:37.179167 21764 leader_election.cc:302] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d9933f641294109bd395addc5ab8c70, 6d397540b57a4663935e8d3c4f5a417e; no voters:
I20260925 07:28:37.179266 21880 raft_consensus.cc:2781] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:28:37.179388 21880 raft_consensus.cc:686] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70 [term 1 LEADER]: Becoming Leader. State: Replica: 3d9933f641294109bd395addc5ab8c70, State: Running, Role: LEADER
I20260925 07:28:37.179466 21880 consensus_queue.cc:237] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98d5e5d89d654dab96fc59e589ef6cfd" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 43065 } } peers { permanent_uuid: "6d397540b57a4663935e8d3c4f5a417e" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 45323 } } peers { permanent_uuid: "3d9933f641294109bd395addc5ab8c70" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 37045 } }
I20260925 07:28:37.180102 21687 catalog_manager.cc:5495] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d9933f641294109bd395addc5ab8c70 (127.5.158.65). New cstate: current_term: 1 leader_uuid: "3d9933f641294109bd395addc5ab8c70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98d5e5d89d654dab96fc59e589ef6cfd" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 43065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d397540b57a4663935e8d3c4f5a417e" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 45323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d9933f641294109bd395addc5ab8c70" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 37045 } health_report { overall_health: HEALTHY } } }
I20260925 07:28:37.197623 21883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:37.195444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56214 (local address 127.5.158.65:37045)
0925 07:28:37.195483 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0925 07:28:37.195485 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:28:37.195492 (+ 7us) server_negotiation.cc:407] Connection header received
0925 07:28:37.195539 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:28:37.195540 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:28:37.195553 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:28:37.195607 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 07:28:37.195722 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:28:37.196181 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:28:37.196858 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:28:37.197067 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:28:37.197153 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:28:37.197319 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 07:28:37.197414 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 07:28:37.197494 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0925 07:28:37.197542 (+ 48us) server_negotiation.cc:299] Negotiation successful
0925 07:28:37.197559 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 07:28:37.198756 21967 raft_consensus.cc:1261] T 59c05f9a574a4b2db0631fc455feedfe P 98d5e5d89d654dab96fc59e589ef6cfd [term 1 FOLLOWER]: Refusing update from remote peer 3d9933f641294109bd395addc5ab8c70: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:28:37.199004 22099 raft_consensus.cc:1261] T 59c05f9a574a4b2db0631fc455feedfe P 6d397540b57a4663935e8d3c4f5a417e [term 1 FOLLOWER]: Refusing update from remote peer 3d9933f641294109bd395addc5ab8c70: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:28:37.199026 21880 consensus_queue.cc:1035] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98d5e5d89d654dab96fc59e589ef6cfd" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 43065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:28:37.199273 21880 consensus_queue.cc:1035] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d397540b57a4663935e8d3c4f5a417e" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 45323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:28:37.200805 22164 mvcc.cc:205] Tried to move back new op lower bound from 7333156115244269568 to 7333156115167490048. Current Snapshot: MvccSnapshot[applied={T|T < 7333156115244269568}]
I20260925 07:28:37.201496 22167 mvcc.cc:205] Tried to move back new op lower bound from 7333156115244269568 to 7333156115167490048. Current Snapshot: MvccSnapshot[applied={T|T < 7333156115244269568}]
I20260925 07:28:37.203135 21687 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45516:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 07:28:37.203212 21687 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45516:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 07:28:37.203791 21687 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de: Sending DeleteTablet for 3 replicas of tablet 59c05f9a574a4b2db0631fc455feedfe
I20260925 07:28:37.204051 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 21725
I20260925 07:28:37.204118 21947 tablet_service.cc:1514] Processing DeleteTablet for tablet 59c05f9a574a4b2db0631fc455feedfe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:28:37 UTC) from {username='kudu', principal='kudu/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:47060
I20260925 07:28:37.204195 22079 tablet_service.cc:1514] Processing DeleteTablet for tablet 59c05f9a574a4b2db0631fc455feedfe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:28:37 UTC) from {username='kudu', principal='kudu/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:52646
I20260925 07:28:37.204372 22174 tablet_replica.cc:330] T 59c05f9a574a4b2db0631fc455feedfe P 6d397540b57a4663935e8d3c4f5a417e: stopping tablet replica
I20260925 07:28:37.204460 22174 raft_consensus.cc:2229] T 59c05f9a574a4b2db0631fc455feedfe P 6d397540b57a4663935e8d3c4f5a417e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 07:28:37.204636 22174 raft_consensus.cc:2258] T 59c05f9a574a4b2db0631fc455feedfe P 6d397540b57a4663935e8d3c4f5a417e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:28:37.204963 22174 ts_tablet_manager.cc:1905] T 59c05f9a574a4b2db0631fc455feedfe P 6d397540b57a4663935e8d3c4f5a417e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:28:37.205580 22173 tablet_replica.cc:330] T 59c05f9a574a4b2db0631fc455feedfe P 98d5e5d89d654dab96fc59e589ef6cfd: stopping tablet replica
I20260925 07:28:37.205667 22173 raft_consensus.cc:2229] T 59c05f9a574a4b2db0631fc455feedfe P 98d5e5d89d654dab96fc59e589ef6cfd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 07:28:37.205760 22173 raft_consensus.cc:2258] T 59c05f9a574a4b2db0631fc455feedfe P 98d5e5d89d654dab96fc59e589ef6cfd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:28:37.206178 22173 ts_tablet_manager.cc:1905] T 59c05f9a574a4b2db0631fc455feedfe P 98d5e5d89d654dab96fc59e589ef6cfd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:28:37.206748 22174 ts_tablet_manager.cc:1918] T 59c05f9a574a4b2db0631fc455feedfe P 6d397540b57a4663935e8d3c4f5a417e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 07:28:37.206815 22174 log.cc:1196] T 59c05f9a574a4b2db0631fc455feedfe P 6d397540b57a4663935e8d3c4f5a417e: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-2/wal/wals/59c05f9a574a4b2db0631fc455feedfe
I20260925 07:28:37.207099 22174 ts_tablet_manager.cc:1939] T 59c05f9a574a4b2db0631fc455feedfe P 6d397540b57a4663935e8d3c4f5a417e: Deleting consensus metadata
I20260925 07:28:37.207472 21674 catalog_manager.cc:4841] TS 6d397540b57a4663935e8d3c4f5a417e (127.5.158.67:45323): tablet 59c05f9a574a4b2db0631fc455feedfe (table test-table [id=1691a5094a1a4a8b8cd2c775f65839bf]) successfully deleted
I20260925 07:28:37.207641 22173 ts_tablet_manager.cc:1918] T 59c05f9a574a4b2db0631fc455feedfe P 98d5e5d89d654dab96fc59e589ef6cfd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 07:28:37.207692 22173 log.cc:1196] T 59c05f9a574a4b2db0631fc455feedfe P 98d5e5d89d654dab96fc59e589ef6cfd: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-1/wal/wals/59c05f9a574a4b2db0631fc455feedfe
I20260925 07:28:37.207934 22173 ts_tablet_manager.cc:1939] T 59c05f9a574a4b2db0631fc455feedfe P 98d5e5d89d654dab96fc59e589ef6cfd: Deleting consensus metadata
I20260925 07:28:37.208292 21672 catalog_manager.cc:4841] TS 98d5e5d89d654dab96fc59e589ef6cfd (127.5.158.66:43065): tablet 59c05f9a574a4b2db0631fc455feedfe (table test-table [id=1691a5094a1a4a8b8cd2c775f65839bf]) successfully deleted
W20260925 07:28:37.209199 21673 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.158.65:37045 (error 108)
I20260925 07:28:37.209534 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 21881
I20260925 07:28:37.209576 22151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:37.209470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:37045 (local address 127.0.0.1:56222)
0925 07:28:37.209512 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:37.209549 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.65:37045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260925 07:28:37.213164 21673 catalog_manager.cc:4584] TS 3d9933f641294109bd395addc5ab8c70 (127.5.158.65:37045): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 59c05f9a574a4b2db0631fc455feedfe: Network error: Client connection negotiation failed: client connection to 127.5.158.65:37045: connect: Connection refused (error 111)
I20260925 07:28:37.214567 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 22016
I20260925 07:28:37.218734 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 21654
I20260925 07:28:57.224400 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:32843
--webserver_interface=127.5.158.126
--webserver_port=44765
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:44845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.158.126:32843
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:28:57.300988 22176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:28:57.301230 22176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:28:57.301256 22176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:28:57.302280 22176 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260925 07:28:57.302311 22176 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260925 07:28:57.302330 22176 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260925 07:28:57.302342 22176 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:28:57.302354 22176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:28:57.302374 22176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:28:57.302392 22176 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:28:57.302403 22176 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:28:57.303716 22176 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:44845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.5.158.126:32843
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:32843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.158.126
--webserver_port=44765
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22176
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:28:57.303908 22176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:28:57.304111 22176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:28:57.308808 22182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:28:57.308828 22181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:28:57.308830 22184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:28:57.308862 22176 server_base.cc:1034] running on GCE node
I20260925 07:28:57.309234 22176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:28:57.309571 22176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:28:57.310724 22176 hybrid_clock.cc:648] HybridClock initialized: now 1790321337310704 us; error 34 us; skew 500 ppm
Sep 25 07:28:57 dist-test-slave-1drq krb5kdc[21111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321337, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:28:57.316314 22176 init.cc:377] Logged in from keytab as kudu/127.5.158.126@KRBTEST.COM (short username kudu)
I20260925 07:28:57.316681 22176 webserver.cc:460] Webserver started at http://127.5.158.126:44765/ using document root <none> and password file <none>
I20260925 07:28:57.316885 22176 fs_manager.cc:359] Metadata directory not provided
I20260925 07:28:57.316946 22176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:28:57.318228 22176 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 07:28:57.318954 22191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:28:57.319103 22176 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 07:28:57.319190 22176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "94c99bc2c8fd47adb7b95b181d9246de"
format_stamp: "Formatted at 2026-09-25 07:28:15 on dist-test-slave-1drq"
server_key: "60aff4a17a032ff097c0baad88203862"
server_key_iv: "1d61bdd82e828a1a192ee19ad2b3e3d9"
server_key_version: "encryptionkey@0"
I20260925 07:28:57.319485 22176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:28:57.329931 22176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:28:57.331648 22176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:28:57.331794 22176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:28:57.335362 22176 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.126:32843
I20260925 07:28:57.335407 22243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.126:32843 every 8 connection(s)
I20260925 07:28:57.335752 22176 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:28:57.337137 22244 sys_catalog.cc:263] Verifying existing consensus state
I20260925 07:28:57.337783 22244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de: Bootstrap starting.
I20260925 07:28:57.338790 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 22176
I20260925 07:28:57.339052 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.65:37045
--local_ip_for_outbound_sockets=127.5.158.65
--tserver_master_addrs=127.5.158.126:32843
--webserver_port=39241
--webserver_interface=127.5.158.65
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab
--principal=kudu/127.5.158.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:44845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 07:28:57.346535 22244 log.cc:824] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de: Log is configured to *not* fsync() on all Append() calls
I20260925 07:28:57.348520 22244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 07:28:57.348830 22244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de: Bootstrap complete.
I20260925 07:28:57.350847 22244 raft_consensus.cc:348] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94c99bc2c8fd47adb7b95b181d9246de" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 32843 } }
I20260925 07:28:57.351056 22244 raft_consensus.cc:729] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94c99bc2c8fd47adb7b95b181d9246de, State: Initialized, Role: FOLLOWER
I20260925 07:28:57.351224 22244 consensus_queue.cc:260] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94c99bc2c8fd47adb7b95b181d9246de" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 32843 } }
I20260925 07:28:57.351296 22244 raft_consensus.cc:388] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 07:28:57.351370 22244 raft_consensus.cc:482] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 07:28:57.351416 22244 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de [term 2 FOLLOWER]: Advancing to term 3
I20260925 07:28:57.352290 22244 raft_consensus.cc:504] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94c99bc2c8fd47adb7b95b181d9246de" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 32843 } }
I20260925 07:28:57.352406 22244 leader_election.cc:302] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 94c99bc2c8fd47adb7b95b181d9246de; no voters:
I20260925 07:28:57.352545 22244 leader_election.cc:288] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de [CANDIDATE]: Term 3 election: Requested vote from peers
I20260925 07:28:57.352624 22248 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de [term 3 FOLLOWER]: Leader election won for term 3
I20260925 07:28:57.352752 22244 sys_catalog.cc:564] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:28:57.352807 22248 raft_consensus.cc:686] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de [term 3 LEADER]: Becoming Leader. State: Replica: 94c99bc2c8fd47adb7b95b181d9246de, State: Running, Role: LEADER
I20260925 07:28:57.352917 22248 consensus_queue.cc:237] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94c99bc2c8fd47adb7b95b181d9246de" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 32843 } }
I20260925 07:28:57.353229 22249 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "94c99bc2c8fd47adb7b95b181d9246de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94c99bc2c8fd47adb7b95b181d9246de" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 32843 } } }
I20260925 07:28:57.353227 22248 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de [sys.catalog]: SysCatalogTable state changed. Reason: New leader 94c99bc2c8fd47adb7b95b181d9246de. Latest consensus state: current_term: 3 leader_uuid: "94c99bc2c8fd47adb7b95b181d9246de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94c99bc2c8fd47adb7b95b181d9246de" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 32843 } } }
I20260925 07:28:57.353438 22248 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de [sys.catalog]: This master's current role is: LEADER
I20260925 07:28:57.353376 22249 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de [sys.catalog]: This master's current role is: LEADER
I20260925 07:28:57.353612 22257 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 07:28:57.354303 22257 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 07:28:57.354396 22257 catalog_manager.cc:1259] Loaded cluster ID: c111839a6f3744caa17ed42fd6e29909
I20260925 07:28:57.354426 22257 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 07:28:57.355717 22257 catalog_manager.cc:1504] Loading token signing keys...
I20260925 07:28:57.355876 22257 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de: Loaded TSK: 0
I20260925 07:28:57.355916 22257 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 94c99bc2c8fd47adb7b95b181d9246de: Loaded TSK: 1
F20260925 07:28:57.356115 22257 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f51271139ad google::LogMessage::Fail() at ??:0
@ 0x7f512711688a google::LogMessage::SendToLog() at ??:0
@ 0x7f51271134de google::LogMessage::Flush() at ??:0
@ 0x7f5127114a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f5129880104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f512987dd41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f512f06d1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f512f06d48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f512f06d594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f512f033c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f512f03dad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f5128fe10d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f5128fd6954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f5129e3a6db start_thread at ??:0
@ 0x7f5125c1161f clone at ??:0
W20260925 07:28:57.436594 22246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:28:57.436757 22246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:28:57.436777 22246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:28:57.437927 22246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:28:57.437968 22246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:28:57.437992 22246 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.65
I20260925 07:28:57.439879 22246 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:44845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab
--principal=kudu/127.5.158.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.65:37045
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.158.65
--webserver_port=39241
--tserver_master_addrs=127.5.158.126:32843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22246
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.65
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:28:57.440115 22246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:28:57.440325 22246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:28:57.444913 22288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:28:57.444975 22285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:28:57.444914 22286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:28:57.445029 22246 server_base.cc:1034] running on GCE node
I20260925 07:28:57.445370 22246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:28:57.445583 22246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:28:57.446755 22246 hybrid_clock.cc:648] HybridClock initialized: now 1790321337446724 us; error 46 us; skew 500 ppm
Sep 25 07:28:57 dist-test-slave-1drq krb5kdc[21111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321337, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:28:57.449333 22246 init.cc:377] Logged in from keytab as kudu/127.5.158.65@KRBTEST.COM (short username kudu)
I20260925 07:28:57.449766 22246 webserver.cc:460] Webserver started at http://127.5.158.65:39241/ using document root <none> and password file <none>
I20260925 07:28:57.449980 22246 fs_manager.cc:359] Metadata directory not provided
I20260925 07:28:57.450050 22246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:28:57.451314 22246 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 07:28:57.451939 22295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:28:57.452068 22246 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 07:28:57.452136 22246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-0/wal
uuid: "3d9933f641294109bd395addc5ab8c70"
format_stamp: "Formatted at 2026-09-25 07:28:16 on dist-test-slave-1drq"
server_key: "a35782025adaf52818d49771bb8a133c"
server_key_iv: "1fc684a67be1a898a1cc4aadc88a85e5"
server_key_version: "encryptionkey@0"
I20260925 07:28:57.452411 22246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:28:57.472904 22246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:28:57.475083 22246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:28:57.475220 22246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:28:57.475448 22246 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:28:57.475855 22302 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260925 07:28:57.477833 22246 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260925 07:28:57.477890 22246 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260925 07:28:57.477931 22246 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260925 07:28:57.478533 22246 ts_tablet_manager.cc:610] Registered 1 tablets
I20260925 07:28:57.478586 22246 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260925 07:28:57.478653 22302 tablet_bootstrap.cc:492] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70: Bootstrap starting.
I20260925 07:28:57.485739 22246 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.65:37045
I20260925 07:28:57.485862 22409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.65:37045 every 8 connection(s)
I20260925 07:28:57.486122 22246 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
I20260925 07:28:57.489494 22302 log.cc:824] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70: Log is configured to *not* fsync() on all Append() calls
I20260925 07:28:57.492199 22302 tablet_bootstrap.cc:492] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 07:28:57.492554 22302 tablet_bootstrap.cc:492] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70: Bootstrap complete.
I20260925 07:28:57.493700 22302 ts_tablet_manager.cc:1397] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.010s
I20260925 07:28:57.494206 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 22246
I20260925 07:28:57.494434 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.66:43065
--local_ip_for_outbound_sockets=127.5.158.66
--tserver_master_addrs=127.5.158.126:32843
--webserver_port=45961
--webserver_interface=127.5.158.66
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab
--principal=kudu/127.5.158.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:44845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 07:28:57.495059 22302 raft_consensus.cc:348] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98d5e5d89d654dab96fc59e589ef6cfd" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 43065 } } peers { permanent_uuid: "6d397540b57a4663935e8d3c4f5a417e" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 45323 } } peers { permanent_uuid: "3d9933f641294109bd395addc5ab8c70" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 37045 } }
I20260925 07:28:57.495400 22302 raft_consensus.cc:729] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d9933f641294109bd395addc5ab8c70, State: Initialized, Role: FOLLOWER
I20260925 07:28:57.495568 22302 consensus_queue.cc:260] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98d5e5d89d654dab96fc59e589ef6cfd" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 43065 } } peers { permanent_uuid: "6d397540b57a4663935e8d3c4f5a417e" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 45323 } } peers { permanent_uuid: "3d9933f641294109bd395addc5ab8c70" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 37045 } }
I20260925 07:28:57.495858 22302 ts_tablet_manager.cc:1428] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
W20260925 07:28:57.545454 22412 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:28:57.486710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.65:58177)
0925 07:28:57.486868 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:57.486908 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 07:28:57.487322 (+ 414us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:28:57.545257 (+ 57935us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:32843: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":88,"threads_started":1}
W20260925 07:28:57.545724 22410 heartbeater.cc:644] Failed to heartbeat to 127.5.158.126:32843 (0 consecutive failures): Network error: Failed to ping master at 127.5.158.126:32843: Client connection negotiation failed: client connection to 127.5.158.126:32843: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 07:28:57.546130 22412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:57.546002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.65:38401)
0925 07:28:57.546057 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:57.546098 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260925 07:28:57.546519 22412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:57.546434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.65:60813)
0925 07:28:57.546479 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:57.546503 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260925 07:28:57.546648 22410 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 07:28:57.574313 22414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:28:57.574508 22414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:28:57.574544 22414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:28:57.575626 22414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:28:57.575687 22414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:28:57.575743 22414 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.66
I20260925 07:28:57.577234 22414 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:44845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab
--principal=kudu/127.5.158.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.66:43065
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.158.66
--webserver_port=45961
--tserver_master_addrs=127.5.158.126:32843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22414
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.66
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:28:57.577462 22414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:28:57.577700 22414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:28:57.582154 22423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:28:57.582177 22420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:28:57.582187 22421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:28:57.582237 22414 server_base.cc:1034] running on GCE node
I20260925 07:28:57.582527 22414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:28:57.582754 22414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:28:57.583911 22414 hybrid_clock.cc:648] HybridClock initialized: now 1790321337583899 us; error 27 us; skew 500 ppm
Sep 25 07:28:57 dist-test-slave-1drq krb5kdc[21111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321337, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:28:57.586395 22414 init.cc:377] Logged in from keytab as kudu/127.5.158.66@KRBTEST.COM (short username kudu)
I20260925 07:28:57.586790 22414 webserver.cc:460] Webserver started at http://127.5.158.66:45961/ using document root <none> and password file <none>
I20260925 07:28:57.587011 22414 fs_manager.cc:359] Metadata directory not provided
I20260925 07:28:57.587080 22414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:28:57.588279 22414 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 07:28:57.588837 22430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:28:57.588995 22414 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:28:57.589071 22414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-1/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-1/wal
uuid: "98d5e5d89d654dab96fc59e589ef6cfd"
format_stamp: "Formatted at 2026-09-25 07:28:16 on dist-test-slave-1drq"
server_key: "ed675f764ddbc1fc0df3b050bb9dfc3a"
server_key_iv: "38b84b7c338f9ff97f9466b778bd7b49"
server_key_version: "encryptionkey@0"
I20260925 07:28:57.589407 22414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:28:57.598233 22414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:28:57.600265 22414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:28:57.600363 22414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:28:57.600514 22414 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:28:57.600713 22414 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:28:57.600739 22414 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:28:57.600757 22414 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:28:57.600769 22414 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:28:57.606231 22414 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.66:43065
I20260925 07:28:57.606305 22543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.66:43065 every 8 connection(s)
I20260925 07:28:57.606564 22414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
I20260925 07:28:57.607400 22546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:57.607098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.66:50161)
0925 07:28:57.607234 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:57.607287 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":71,"threads_started":1}
W20260925 07:28:57.607748 22544 heartbeater.cc:644] Failed to heartbeat to 127.5.158.126:32843 (0 consecutive failures): Network error: Failed to ping master at 127.5.158.126:32843: Client connection negotiation failed: client connection to 127.5.158.126:32843: connect: Connection refused (error 111)
I20260925 07:28:57.608018 22546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:57.607927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.66:38269)
0925 07:28:57.607974 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:57.607997 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260925 07:28:57.608283 22546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:57.608202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.66:58973)
0925 07:28:57.608244 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:57.608265 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260925 07:28:57.608385 22544 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260925 07:28:57.609280 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 22414
I20260925 07:28:57.609516 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.67:45323
--local_ip_for_outbound_sockets=127.5.158.67
--tserver_master_addrs=127.5.158.126:32843
--webserver_port=38377
--webserver_interface=127.5.158.67
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab
--principal=kudu/127.5.158.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:44845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:28:57.681737 22547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:28:57.681929 22547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:28:57.681962 22547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:28:57.682976 22547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:28:57.683038 22547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:28:57.683079 22547 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.67
I20260925 07:28:57.684366 22547 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:44845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab
--principal=kudu/127.5.158.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.67:45323
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.158.67
--webserver_port=38377
--tserver_master_addrs=127.5.158.126:32843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22547
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.67
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:28:57.684592 22547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:28:57.684803 22547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:28:57.689339 22552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:28:57.689416 22547 server_base.cc:1034] running on GCE node
W20260925 07:28:57.689327 22555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:28:57.689342 22553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:28:57.689719 22547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:28:57.689945 22547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:28:57.691109 22547 hybrid_clock.cc:648] HybridClock initialized: now 1790321337691104 us; error 34 us; skew 500 ppm
Sep 25 07:28:57 dist-test-slave-1drq krb5kdc[21111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321337, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:28:57.693706 22547 init.cc:377] Logged in from keytab as kudu/127.5.158.67@KRBTEST.COM (short username kudu)
I20260925 07:28:57.694023 22547 webserver.cc:460] Webserver started at http://127.5.158.67:38377/ using document root <none> and password file <none>
I20260925 07:28:57.694181 22547 fs_manager.cc:359] Metadata directory not provided
I20260925 07:28:57.694216 22547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:28:57.695241 22547 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 07:28:57.695727 22562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:28:57.695884 22547 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 07:28:57.695976 22547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-2/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-2/wal
uuid: "6d397540b57a4663935e8d3c4f5a417e"
format_stamp: "Formatted at 2026-09-25 07:28:16 on dist-test-slave-1drq"
server_key: "38f38507c7e2c77c84af8fd08ed34aa4"
server_key_iv: "2704311d537a11eb8df9d149c48cce82"
server_key_version: "encryptionkey@0"
I20260925 07:28:57.696244 22547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:28:57.711973 22547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:28:57.714311 22547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:28:57.714480 22547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:28:57.714704 22547 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:28:57.715006 22547 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:28:57.715067 22547 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:28:57.715109 22547 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:28:57.715142 22547 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:28:57.721277 22547 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.67:45323
I20260925 07:28:57.721308 22675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.67:45323 every 8 connection(s)
I20260925 07:28:57.721730 22547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
I20260925 07:28:57.722690 22678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:57.722395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.67:55259)
0925 07:28:57.722518 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:57.722574 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":58,"threads_started":1}
W20260925 07:28:57.723032 22676 heartbeater.cc:644] Failed to heartbeat to 127.5.158.126:32843 (0 consecutive failures): Network error: Failed to ping master at 127.5.158.126:32843: Client connection negotiation failed: client connection to 127.5.158.126:32843: connect: Connection refused (error 111)
I20260925 07:28:57.723279 22678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:57.723200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.67:41175)
0925 07:28:57.723239 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:57.723264 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260925 07:28:57.723618 22678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:57.723499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.67:38419)
0925 07:28:57.723568 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:57.723597 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260925 07:28:57.723758 22676 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260925 07:28:57.723981 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 22547
I20260925 07:28:58.547139 22680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:58.546939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.65:40467)
0925 07:28:58.547047 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:58.547081 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":59,"threads_started":1}
I20260925 07:28:58.608812 22681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:58.608641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.66:50317)
0925 07:28:58.608736 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:58.608767 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":49,"threads_started":1}
I20260925 07:28:58.724388 22682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:58.724228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.67:44257)
0925 07:28:58.724317 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:58.724345 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77,"thread_start_us":45,"threads_started":1}
I20260925 07:28:59.323796 22683 raft_consensus.cc:482] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 07:28:59.323958 22683 raft_consensus.cc:504] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98d5e5d89d654dab96fc59e589ef6cfd" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 43065 } } peers { permanent_uuid: "6d397540b57a4663935e8d3c4f5a417e" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 45323 } } peers { permanent_uuid: "3d9933f641294109bd395addc5ab8c70" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 37045 } }
I20260925 07:28:59.324325 22683 leader_election.cc:288] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 98d5e5d89d654dab96fc59e589ef6cfd (127.5.158.66:43065), 6d397540b57a4663935e8d3c4f5a417e (127.5.158.67:45323)
Sep 25 07:28:59 dist-test-slave-1drq krb5kdc[21111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321337, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.65@KRBTEST.COM for kudu/127.5.158.66@KRBTEST.COM
Sep 25 07:28:59 dist-test-slave-1drq krb5kdc[21111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321337, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.65@KRBTEST.COM for kudu/127.5.158.67@KRBTEST.COM
I20260925 07:28:59.336607 22685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:59.324506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.67:45323 (local address 127.5.158.65:44673)
0925 07:28:59.324668 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:59.324681 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 07:28:59.324742 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:28:59.325758 (+ 1016us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:28:59.325762 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:28:59.326318 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:28:59.326590 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:28:59.326598 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:28:59.328206 (+ 1608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:28:59.328207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:28:59.329496 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:28:59.329500 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:28:59.330289 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:28:59.330290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:28:59.330427 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:28:59.331114 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:28:59.331125 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:28:59.333292 (+ 2167us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:28:59.335279 (+ 1987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:28:59.335280 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:28:59.335280 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:28:59.335426 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:28:59.335684 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:28:59.335684 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:28:59.335685 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:28:59.335726 (+ 41us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:28:59.336083 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:28:59.336083 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:28:59.336169 (+ 86us) client_negotiation.cc:769] Sending connection context
0925 07:28:59.336285 (+ 116us) client_negotiation.cc:240] Negotiation successful
0925 07:28:59.336467 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":75,"thread_start_us":74,"threads_started":1}
I20260925 07:28:59.336616 22686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:59.324619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:53801 (local address 127.5.158.66:43065)
0925 07:28:59.324764 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0925 07:28:59.324777 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 07:28:59.324801 (+ 24us) server_negotiation.cc:407] Connection header received
0925 07:28:59.325419 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:28:59.325422 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:28:59.325499 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:28:59.325652 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:28:59.326717 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:28:59.328082 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:28:59.329580 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:28:59.330172 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:28:59.330291 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:28:59.333042 (+ 2751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:28:59.333046 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:28:59.333047 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:28:59.333059 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:28:59.334950 (+ 1891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:28:59.335328 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:28:59.335330 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:28:59.335331 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:28:59.335410 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:28:59.335634 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:28:59.335635 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:28:59.335636 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:28:59.335951 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:28:59.336037 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0925 07:28:59.336359 (+ 322us) server_negotiation.cc:299] Negotiation successful
0925 07:28:59.336500 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":47,"threads_started":1}
I20260925 07:28:59.336616 22687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:59.324682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:44673 (local address 127.5.158.67:45323)
0925 07:28:59.324808 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0925 07:28:59.324815 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 07:28:59.324830 (+ 15us) server_negotiation.cc:407] Connection header received
0925 07:28:59.325419 (+ 589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:28:59.325422 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:28:59.325499 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:28:59.325652 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:28:59.326717 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:28:59.328082 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:28:59.329583 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:28:59.330169 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:28:59.330292 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:28:59.333420 (+ 3128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:28:59.333424 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:28:59.333424 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:28:59.333435 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:28:59.335212 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:28:59.335519 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:28:59.335521 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:28:59.335521 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:28:59.335614 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:28:59.335783 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:28:59.335784 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:28:59.335784 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:28:59.336024 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:28:59.336092 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 07:28:59.336366 (+ 274us) server_negotiation.cc:299] Negotiation successful
0925 07:28:59.336500 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":49,"threads_started":1}
I20260925 07:28:59.336617 22684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:59.324506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.66:43065 (local address 127.5.158.65:53801)
0925 07:28:59.324670 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:59.324681 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 07:28:59.324742 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:28:59.325758 (+ 1016us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:28:59.325762 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:28:59.326323 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:28:59.326590 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:28:59.326597 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:28:59.328206 (+ 1609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:28:59.328207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:28:59.329496 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:28:59.329500 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:28:59.330288 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:28:59.330289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:28:59.330427 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:28:59.331114 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:28:59.331126 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:28:59.332915 (+ 1789us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:28:59.335027 (+ 2112us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:28:59.335038 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:28:59.335040 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:28:59.335225 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:28:59.335472 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:28:59.335472 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:28:59.335473 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:28:59.335573 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:28:59.336018 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:28:59.336023 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:28:59.336169 (+ 146us) client_negotiation.cc:769] Sending connection context
0925 07:28:59.336285 (+ 116us) client_negotiation.cc:240] Negotiation successful
0925 07:28:59.336537 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":167,"thread_start_us":76,"threads_started":1}
W20260925 07:28:59.338160 22297 leader_election.cc:341] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 98d5e5d89d654dab96fc59e589ef6cfd (127.5.158.66:43065): Not found: Tablet not found: 59c05f9a574a4b2db0631fc455feedfe
W20260925 07:28:59.338253 22299 leader_election.cc:341] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6d397540b57a4663935e8d3c4f5a417e (127.5.158.67:45323): Not found: Tablet not found: 59c05f9a574a4b2db0631fc455feedfe
I20260925 07:28:59.338332 22299 leader_election.cc:302] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d9933f641294109bd395addc5ab8c70; no voters: 6d397540b57a4663935e8d3c4f5a417e, 98d5e5d89d654dab96fc59e589ef6cfd
I20260925 07:28:59.338471 22683 raft_consensus.cc:2726] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 07:28:59.547751 22684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:59.547528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.65:50425)
0925 07:28:59.547657 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:59.547692 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260925 07:28:59.609443 22688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:59.609252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.66:59791)
0925 07:28:59.609366 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:59.609405 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":50,"threads_started":1}
I20260925 07:28:59.724931 22689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:28:59.724732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.67:36353)
0925 07:28:59.724856 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0925 07:28:59.724895 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":65,"threads_started":1}
I20260925 07:29:00.548496 22690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:00.548281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.65:59769)
0925 07:29:00.548406 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:00.548449 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":61,"threads_started":1}
I20260925 07:29:00.610008 22691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:00.609809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.66:53467)
0925 07:29:00.609930 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:00.609967 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":49,"threads_started":1}
I20260925 07:29:00.725615 22692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:00.725398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.67:33843)
0925 07:29:00.725546 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:00.725583 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":90,"threads_started":1}
I20260925 07:29:01.051345 22693 raft_consensus.cc:482] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 07:29:01.051427 22693 raft_consensus.cc:504] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98d5e5d89d654dab96fc59e589ef6cfd" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 43065 } } peers { permanent_uuid: "6d397540b57a4663935e8d3c4f5a417e" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 45323 } } peers { permanent_uuid: "3d9933f641294109bd395addc5ab8c70" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 37045 } }
I20260925 07:29:01.051635 22693 leader_election.cc:288] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 98d5e5d89d654dab96fc59e589ef6cfd (127.5.158.66:43065), 6d397540b57a4663935e8d3c4f5a417e (127.5.158.67:45323)
W20260925 07:29:01.052060 22297 leader_election.cc:341] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 98d5e5d89d654dab96fc59e589ef6cfd (127.5.158.66:43065): Not found: Tablet not found: 59c05f9a574a4b2db0631fc455feedfe
W20260925 07:29:01.052132 22299 leader_election.cc:341] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6d397540b57a4663935e8d3c4f5a417e (127.5.158.67:45323): Not found: Tablet not found: 59c05f9a574a4b2db0631fc455feedfe
I20260925 07:29:01.052183 22299 leader_election.cc:302] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d9933f641294109bd395addc5ab8c70; no voters: 6d397540b57a4663935e8d3c4f5a417e, 98d5e5d89d654dab96fc59e589ef6cfd
I20260925 07:29:01.052254 22693 raft_consensus.cc:2726] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 07:29:01.549196 22694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:01.548948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.65:41963)
0925 07:29:01.549100 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:01.549132 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":64,"threads_started":1}
I20260925 07:29:01.610535 22695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:01.610373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.66:40739)
0925 07:29:01.610480 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:01.610504 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":66,"threads_started":1}
I20260925 07:29:01.726150 22696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:01.725960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.67:49911)
0925 07:29:01.726076 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:01.726106 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":65,"threads_started":1}
I20260925 07:29:02.549826 22697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:02.549641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.65:36055)
0925 07:29:02.549757 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:02.549784 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":71,"threads_started":1}
I20260925 07:29:02.611083 22698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:02.610872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.66:49307)
0925 07:29:02.610983 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:02.611029 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":56,"threads_started":1}
I20260925 07:29:02.726670 22699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:02.726507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.67:42147)
0925 07:29:02.726608 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:02.726637 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":51,"threads_started":1}
I20260925 07:29:02.853580 22700 raft_consensus.cc:482] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 07:29:02.853667 22700 raft_consensus.cc:504] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98d5e5d89d654dab96fc59e589ef6cfd" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 43065 } } peers { permanent_uuid: "6d397540b57a4663935e8d3c4f5a417e" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 45323 } } peers { permanent_uuid: "3d9933f641294109bd395addc5ab8c70" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 37045 } }
I20260925 07:29:02.853839 22700 leader_election.cc:288] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 98d5e5d89d654dab96fc59e589ef6cfd (127.5.158.66:43065), 6d397540b57a4663935e8d3c4f5a417e (127.5.158.67:45323)
W20260925 07:29:02.854203 22299 leader_election.cc:341] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6d397540b57a4663935e8d3c4f5a417e (127.5.158.67:45323): Not found: Tablet not found: 59c05f9a574a4b2db0631fc455feedfe
W20260925 07:29:02.854298 22297 leader_election.cc:341] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 98d5e5d89d654dab96fc59e589ef6cfd (127.5.158.66:43065): Not found: Tablet not found: 59c05f9a574a4b2db0631fc455feedfe
I20260925 07:29:02.854331 22297 leader_election.cc:302] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d9933f641294109bd395addc5ab8c70; no voters: 6d397540b57a4663935e8d3c4f5a417e, 98d5e5d89d654dab96fc59e589ef6cfd
I20260925 07:29:02.854419 22700 raft_consensus.cc:2726] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 07:29:03.550429 22701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:03.550232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.65:35039)
0925 07:29:03.550354 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:03.550384 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":69,"threads_started":1}
I20260925 07:29:03.611937 22702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:03.611512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.66:53625)
0925 07:29:03.611815 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:03.611848 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":84,"threads_started":1}
I20260925 07:29:03.727257 22703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:03.727089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.67:50127)
0925 07:29:03.727196 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:03.727224 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":49,"threads_started":1}
I20260925 07:29:04.551116 22704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:04.550917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.65:36531)
0925 07:29:04.551044 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:04.551074 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":69,"threads_started":1}
I20260925 07:29:04.612519 22705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:04.612323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.66:42593)
0925 07:29:04.612450 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:04.612486 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":61,"threads_started":1}
I20260925 07:29:04.727808 22706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:04.727616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.67:58211)
0925 07:29:04.727737 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:04.727773 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":61,"threads_started":1}
I20260925 07:29:05.551754 22707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:05.551544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.65:45871)
0925 07:29:05.551685 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:05.551711 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":91,"threads_started":1}
I20260925 07:29:05.613042 22708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:05.612889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.66:48217)
0925 07:29:05.612983 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:05.613011 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82,"thread_start_us":52,"threads_started":1}
I20260925 07:29:05.728528 22709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:05.728310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.67:38963)
0925 07:29:05.728454 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:05.728490 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":69,"threads_started":1}
I20260925 07:29:06.193634 22710 raft_consensus.cc:482] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 07:29:06.193728 22710 raft_consensus.cc:504] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98d5e5d89d654dab96fc59e589ef6cfd" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 43065 } } peers { permanent_uuid: "6d397540b57a4663935e8d3c4f5a417e" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 45323 } } peers { permanent_uuid: "3d9933f641294109bd395addc5ab8c70" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 37045 } }
I20260925 07:29:06.193914 22710 leader_election.cc:288] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 98d5e5d89d654dab96fc59e589ef6cfd (127.5.158.66:43065), 6d397540b57a4663935e8d3c4f5a417e (127.5.158.67:45323)
W20260925 07:29:06.194274 22297 leader_election.cc:341] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 98d5e5d89d654dab96fc59e589ef6cfd (127.5.158.66:43065): Not found: Tablet not found: 59c05f9a574a4b2db0631fc455feedfe
W20260925 07:29:06.194352 22299 leader_election.cc:341] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6d397540b57a4663935e8d3c4f5a417e (127.5.158.67:45323): Not found: Tablet not found: 59c05f9a574a4b2db0631fc455feedfe
I20260925 07:29:06.194469 22299 leader_election.cc:302] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d9933f641294109bd395addc5ab8c70; no voters: 6d397540b57a4663935e8d3c4f5a417e, 98d5e5d89d654dab96fc59e589ef6cfd
I20260925 07:29:06.194547 22710 raft_consensus.cc:2726] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 07:29:06.552390 22711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:06.552135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.65:37723)
0925 07:29:06.552317 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:06.552349 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":76,"threads_started":1}
I20260925 07:29:06.613546 22712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:06.613393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.66:39111)
0925 07:29:06.613489 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:06.613515 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":52,"threads_started":1}
I20260925 07:29:06.729182 22713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:06.728942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.67:40205)
0925 07:29:06.729059 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:06.729139 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":67,"threads_started":1}
I20260925 07:29:07.553074 22714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:07.552852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.65:42891)
0925 07:29:07.552996 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:07.553030 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":67,"threads_started":1}
I20260925 07:29:07.614056 22715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:07.613895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.66:37801)
0925 07:29:07.613998 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:07.614024 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":60,"threads_started":1}
I20260925 07:29:07.729800 22716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:07.729579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.67:55813)
0925 07:29:07.729738 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:07.729767 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":81,"threads_started":1}
I20260925 07:29:08.553892 22717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:08.553696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.65:35847)
0925 07:29:08.553818 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:08.553851 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":71,"threads_started":1}
I20260925 07:29:08.614707 22718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:08.614501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.66:54585)
0925 07:29:08.614647 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:08.614675 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":83,"threads_started":1}
I20260925 07:29:08.730338 22719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:08.730170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.67:57377)
0925 07:29:08.730280 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:08.730308 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":53,"threads_started":1}
I20260925 07:29:09.554615 22720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:09.554358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.65:34001)
0925 07:29:09.554495 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:09.554560 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":73,"threads_started":1}
I20260925 07:29:09.615339 22721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:09.615160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.66:34587)
0925 07:29:09.615274 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:09.615306 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":59,"threads_started":1}
I20260925 07:29:09.730995 22722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:09.730823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.67:35047)
0925 07:29:09.730931 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:09.730963 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":52,"threads_started":1}
I20260925 07:29:09.990556 22723 raft_consensus.cc:482] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 07:29:09.990639 22723 raft_consensus.cc:504] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98d5e5d89d654dab96fc59e589ef6cfd" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 43065 } } peers { permanent_uuid: "6d397540b57a4663935e8d3c4f5a417e" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 45323 } } peers { permanent_uuid: "3d9933f641294109bd395addc5ab8c70" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 37045 } }
I20260925 07:29:09.990828 22723 leader_election.cc:288] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 98d5e5d89d654dab96fc59e589ef6cfd (127.5.158.66:43065), 6d397540b57a4663935e8d3c4f5a417e (127.5.158.67:45323)
W20260925 07:29:09.991204 22299 leader_election.cc:341] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6d397540b57a4663935e8d3c4f5a417e (127.5.158.67:45323): Not found: Tablet not found: 59c05f9a574a4b2db0631fc455feedfe
W20260925 07:29:09.991333 22297 leader_election.cc:341] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 98d5e5d89d654dab96fc59e589ef6cfd (127.5.158.66:43065): Not found: Tablet not found: 59c05f9a574a4b2db0631fc455feedfe
I20260925 07:29:09.991408 22297 leader_election.cc:302] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d9933f641294109bd395addc5ab8c70; no voters: 6d397540b57a4663935e8d3c4f5a417e, 98d5e5d89d654dab96fc59e589ef6cfd
I20260925 07:29:09.991509 22723 raft_consensus.cc:2726] T 59c05f9a574a4b2db0631fc455feedfe P 3d9933f641294109bd395addc5ab8c70 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 07:29:10.555183 22724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:10.555008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.65:59895)
0925 07:29:10.555112 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:10.555142 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":54,"threads_started":1}
I20260925 07:29:10.615953 22725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:10.615777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.66:51603)
0925 07:29:10.615893 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:10.615918 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":70,"threads_started":1}
I20260925 07:29:10.731642 22726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:10.731407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.67:47629)
0925 07:29:10.731565 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:10.731598 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":92,"threads_started":1}
I20260925 07:29:11.555737 22727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:11.555535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.65:48189)
0925 07:29:11.555647 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:11.555692 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":56,"threads_started":1}
I20260925 07:29:11.616485 22728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:11.616308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.66:51745)
0925 07:29:11.616417 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:11.616451 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":55,"threads_started":1}
I20260925 07:29:11.732313 22729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:11.732032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.67:49019)
0925 07:29:11.732234 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:11.732266 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":50,"threads_started":1}
I20260925 07:29:12.556463 22730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:12.556284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.65:50347)
0925 07:29:12.556394 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:12.556425 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":53,"threads_started":1}
I20260925 07:29:12.617029 22731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:12.616856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.66:35463)
0925 07:29:12.616971 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:12.617000 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":58,"threads_started":1}
I20260925 07:29:12.726902 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 22246
I20260925 07:29:12.731384 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 22414
I20260925 07:29:12.732925 22732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:12.732767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:32843 (local address 127.5.158.67:47145)
0925 07:29:12.732871 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:12.732896 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.126:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":61,"threads_started":1}
I20260925 07:29:12.735414 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 22547
2026-09-25T07:29:12Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (57951 ms)
[----------] 30 tests from SecurityITest (242419 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 07:29:12 dist-test-slave-1drq krb5kdc[22735](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 07:29:12 dist-test-slave-1drq krb5kdc[22735](info): set up 2 sockets
Sep 25 07:29:12 dist-test-slave-1drq krb5kdc[22735](info): commencing operation
krb5kdc: starting...
W20260925 07:29:13.688505 5753 mini_kdc.cc:121] Time spent starting KDC: real 0.934s user 0.001s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Sep 25 07:29:13 dist-test-slave-1drq krb5kdc[22735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321353, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T07:29:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T07:29:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
I20260925 07:29:13.792268 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:39809
--webserver_interface=127.5.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:40459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.158.126:39809
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:29:13.863260 22748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:29:13.863411 22748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:29:13.863430 22748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:29:13.864323 22748 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:29:13.864351 22748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:29:13.864363 22748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:29:13.864374 22748 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:29:13.864384 22748 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:29:13.865638 22748 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:40459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.158.126:39809
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:39809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22748
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:29:13.865798 22748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:29:13.865929 22748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 07:29:13.870424 22748 server_base.cc:1034] running on GCE node
W20260925 07:29:13.870383 22753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:13.870395 22756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:13.870369 22754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:13.870872 22748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:29:13.871140 22748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:29:13.872272 22748 hybrid_clock.cc:648] HybridClock initialized: now 1790321353872255 us; error 34 us; skew 500 ppm
Sep 25 07:29:13 dist-test-slave-1drq krb5kdc[22735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321353, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:29:13.874390 22748 init.cc:377] Logged in from keytab as kudu/127.5.158.126@KRBTEST.COM (short username kudu)
I20260925 07:29:13.874822 22748 webserver.cc:460] Webserver started at http://127.5.158.126:38527/ using document root <none> and password file <none>
I20260925 07:29:13.875025 22748 fs_manager.cc:359] Metadata directory not provided
I20260925 07:29:13.875062 22748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:29:13.875222 22748 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:29:13.876178 22748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/master-0/data/instance:
uuid: "40a36ac07bfd4dda9396d80a11b28510"
format_stamp: "Formatted at 2026-09-25 07:29:13 on dist-test-slave-1drq"
server_key: "2cb58c93c66865b5e8e9b6500a83f7c9"
server_key_iv: "54b4077b25be7501c7942594077c8fac"
server_key_version: "encryptionkey@0"
I20260925 07:29:13.876484 22748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/master-0/wal/instance:
uuid: "40a36ac07bfd4dda9396d80a11b28510"
format_stamp: "Formatted at 2026-09-25 07:29:13 on dist-test-slave-1drq"
server_key: "2cb58c93c66865b5e8e9b6500a83f7c9"
server_key_iv: "54b4077b25be7501c7942594077c8fac"
server_key_version: "encryptionkey@0"
I20260925 07:29:13.877702 22748 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 07:29:13.878295 22763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:13.878448 22748 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:29:13.878535 22748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "40a36ac07bfd4dda9396d80a11b28510"
format_stamp: "Formatted at 2026-09-25 07:29:13 on dist-test-slave-1drq"
server_key: "2cb58c93c66865b5e8e9b6500a83f7c9"
server_key_iv: "54b4077b25be7501c7942594077c8fac"
server_key_version: "encryptionkey@0"
I20260925 07:29:13.878602 22748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:29:13.896581 22748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:29:13.898109 22748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:29:13.898272 22748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:29:13.902168 22748 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.126:39809
I20260925 07:29:13.902194 22815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.126:39809 every 8 connection(s)
I20260925 07:29:13.902484 22748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:29:13.903226 22816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:29:13.905210 22816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40a36ac07bfd4dda9396d80a11b28510: Bootstrap starting.
I20260925 07:29:13.905697 22816 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 40a36ac07bfd4dda9396d80a11b28510: Neither blocks nor log segments found. Creating new log.
I20260925 07:29:13.906005 22816 log.cc:824] T 00000000000000000000000000000000 P 40a36ac07bfd4dda9396d80a11b28510: Log is configured to *not* fsync() on all Append() calls
I20260925 07:29:13.906016 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 22748
I20260925 07:29:13.906105 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/master-0/wal/instance
I20260925 07:29:13.906352 5753 external_mini_cluster.cc:1428] Setting key 069fa6b9ec424f9fc2c39c7a20a9dde3
I20260925 07:29:13.906865 22816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40a36ac07bfd4dda9396d80a11b28510: No bootstrap required, opened a new log
I20260925 07:29:13.908838 22816 raft_consensus.cc:348] T 00000000000000000000000000000000 P 40a36ac07bfd4dda9396d80a11b28510 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40a36ac07bfd4dda9396d80a11b28510" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 39809 } }
I20260925 07:29:13.909026 22816 raft_consensus.cc:374] T 00000000000000000000000000000000 P 40a36ac07bfd4dda9396d80a11b28510 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:29:13.909065 22816 raft_consensus.cc:729] T 00000000000000000000000000000000 P 40a36ac07bfd4dda9396d80a11b28510 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40a36ac07bfd4dda9396d80a11b28510, State: Initialized, Role: FOLLOWER
I20260925 07:29:13.909235 22816 consensus_queue.cc:260] T 00000000000000000000000000000000 P 40a36ac07bfd4dda9396d80a11b28510 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40a36ac07bfd4dda9396d80a11b28510" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 39809 } }
I20260925 07:29:13.909315 22816 raft_consensus.cc:388] T 00000000000000000000000000000000 P 40a36ac07bfd4dda9396d80a11b28510 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 07:29:13.909344 22816 raft_consensus.cc:482] T 00000000000000000000000000000000 P 40a36ac07bfd4dda9396d80a11b28510 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 07:29:13.909389 22816 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 40a36ac07bfd4dda9396d80a11b28510 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:29:13.910017 22816 raft_consensus.cc:504] T 00000000000000000000000000000000 P 40a36ac07bfd4dda9396d80a11b28510 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40a36ac07bfd4dda9396d80a11b28510" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 39809 } }
I20260925 07:29:13.910130 22816 leader_election.cc:302] T 00000000000000000000000000000000 P 40a36ac07bfd4dda9396d80a11b28510 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 40a36ac07bfd4dda9396d80a11b28510; no voters:
I20260925 07:29:13.910275 22816 leader_election.cc:288] T 00000000000000000000000000000000 P 40a36ac07bfd4dda9396d80a11b28510 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 07:29:13.910328 22821 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 40a36ac07bfd4dda9396d80a11b28510 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:29:13.910456 22816 sys_catalog.cc:564] T 00000000000000000000000000000000 P 40a36ac07bfd4dda9396d80a11b28510 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:29:13.910497 22821 raft_consensus.cc:686] T 00000000000000000000000000000000 P 40a36ac07bfd4dda9396d80a11b28510 [term 1 LEADER]: Becoming Leader. State: Replica: 40a36ac07bfd4dda9396d80a11b28510, State: Running, Role: LEADER
I20260925 07:29:13.910642 22821 consensus_queue.cc:237] T 00000000000000000000000000000000 P 40a36ac07bfd4dda9396d80a11b28510 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40a36ac07bfd4dda9396d80a11b28510" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 39809 } }
I20260925 07:29:13.911636 22823 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40a36ac07bfd4dda9396d80a11b28510 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 40a36ac07bfd4dda9396d80a11b28510. Latest consensus state: current_term: 1 leader_uuid: "40a36ac07bfd4dda9396d80a11b28510" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40a36ac07bfd4dda9396d80a11b28510" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 39809 } } }
I20260925 07:29:13.911711 22823 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40a36ac07bfd4dda9396d80a11b28510 [sys.catalog]: This master's current role is: LEADER
I20260925 07:29:13.911945 22822 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40a36ac07bfd4dda9396d80a11b28510 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "40a36ac07bfd4dda9396d80a11b28510" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40a36ac07bfd4dda9396d80a11b28510" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 39809 } } }
I20260925 07:29:13.912009 22822 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40a36ac07bfd4dda9396d80a11b28510 [sys.catalog]: This master's current role is: LEADER
W20260925 07:29:13.912320 22836 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 40a36ac07bfd4dda9396d80a11b28510: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 07:29:13.912374 22836 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 07:29:13.912421 22837 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 07:29:13.912570 22837 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 25 07:29:13 dist-test-slave-1drq krb5kdc[22735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321353, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:29:13.914752 22837 catalog_manager.cc:1347] Generated new cluster ID: 5f90c40d0a8a429f8da4ff3a3fa6edca
I20260925 07:29:13.914808 22837 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 07:29:13.917555 22820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:13.906840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45520 (local address 127.5.158.126:39809)
0925 07:29:13.907017 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:13.907026 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:13.907089 (+ 63us) server_negotiation.cc:407] Connection header received
0925 07:29:13.908046 (+ 957us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:13.908054 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:13.908113 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:13.908255 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:13.908793 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:13.910447 (+ 1654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:13.911454 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:13.912141 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:13.912235 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:13.914735 (+ 2500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:13.914737 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:13.914738 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:29:13.914746 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:13.916243 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:13.916590 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:13.916592 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:13.916593 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:13.916667 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:13.916845 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:13.916846 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:13.916847 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:13.917043 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:13.917217 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:13.917286 (+ 69us) server_negotiation.cc:299] Negotiation successful
0925 07:29:13.917398 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":73,"threads_started":1}
I20260925 07:29:13.954511 22837 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 07:29:13.954942 22837 catalog_manager.cc:1504] Loading token signing keys...
I20260925 07:29:13.961512 22837 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 40a36ac07bfd4dda9396d80a11b28510: Generated new TSK 0
I20260925 07:29:13.961671 22837 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.158.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab.
Entry for principal kudu/127.5.158.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab.
I20260925 07:29:13.986086 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.65:0
--local_ip_for_outbound_sockets=127.5.158.65
--webserver_interface=127.5.158.65
--webserver_port=0
--tserver_master_addrs=127.5.158.126:39809
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab
--principal=kudu/127.5.158.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:40459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:29:14.056651 22842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:29:14.056799 22842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:29:14.056818 22842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:29:14.057884 22842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:29:14.057924 22842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:29:14.057946 22842 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.65
I20260925 07:29:14.059269 22842 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:40459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab
--principal=kudu/127.5.158.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.158.65
--webserver_port=0
--tserver_master_addrs=127.5.158.126:39809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22842
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.65
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:29:14.059505 22842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:29:14.059743 22842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:29:14.064153 22848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:14.064183 22850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:14.064173 22847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:14.064275 22842 server_base.cc:1034] running on GCE node
I20260925 07:29:14.064507 22842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:29:14.064719 22842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:29:14.065850 22842 hybrid_clock.cc:648] HybridClock initialized: now 1790321354065838 us; error 26 us; skew 500 ppm
Sep 25 07:29:14 dist-test-slave-1drq krb5kdc[22735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321354, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:29:14.068104 22842 init.cc:377] Logged in from keytab as kudu/127.5.158.65@KRBTEST.COM (short username kudu)
I20260925 07:29:14.068514 22842 webserver.cc:460] Webserver started at http://127.5.158.65:33935/ using document root <none> and password file <none>
I20260925 07:29:14.068734 22842 fs_manager.cc:359] Metadata directory not provided
I20260925 07:29:14.068775 22842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:29:14.068924 22842 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:29:14.070001 22842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/ts-0/data/instance:
uuid: "2a45993f77334eda9966ba3763ff91a4"
format_stamp: "Formatted at 2026-09-25 07:29:14 on dist-test-slave-1drq"
server_key: "0d56ac37a35d00fefbc7161037fc1669"
server_key_iv: "a97d9c68845b160d3dcd9a93719d2d09"
server_key_version: "encryptionkey@0"
I20260925 07:29:14.070326 22842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/ts-0/wal/instance:
uuid: "2a45993f77334eda9966ba3763ff91a4"
format_stamp: "Formatted at 2026-09-25 07:29:14 on dist-test-slave-1drq"
server_key: "0d56ac37a35d00fefbc7161037fc1669"
server_key_iv: "a97d9c68845b160d3dcd9a93719d2d09"
server_key_version: "encryptionkey@0"
I20260925 07:29:14.071446 22842 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 07:29:14.072088 22857 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:14.072256 22842 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:29:14.072340 22842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/ts-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/ts-0/wal
uuid: "2a45993f77334eda9966ba3763ff91a4"
format_stamp: "Formatted at 2026-09-25 07:29:14 on dist-test-slave-1drq"
server_key: "0d56ac37a35d00fefbc7161037fc1669"
server_key_iv: "a97d9c68845b160d3dcd9a93719d2d09"
server_key_version: "encryptionkey@0"
I20260925 07:29:14.072412 22842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:29:14.083787 22842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:29:14.085927 22842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:29:14.086066 22842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:29:14.086243 22842 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:29:14.086503 22842 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:29:14.086552 22842 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:14.086587 22842 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:29:14.086612 22842 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:14.091816 22842 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.65:43925
I20260925 07:29:14.091873 22970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.65:43925 every 8 connection(s)
I20260925 07:29:14.092147 22842 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
Sep 25 07:29:14 dist-test-slave-1drq krb5kdc[22735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321354, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.65@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:29:14.099640 22820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:14.092820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:38151 (local address 127.5.158.126:39809)
0925 07:29:14.092857 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:14.092859 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:14.093034 (+ 175us) server_negotiation.cc:407] Connection header received
0925 07:29:14.093445 (+ 411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:14.093445 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:14.093454 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:14.093515 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:14.094268 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:14.094718 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:14.095533 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:14.095707 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:14.095751 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:14.097389 (+ 1638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:14.097390 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:14.097390 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:29:14.097395 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:14.098626 (+ 1231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:14.098913 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:14.098914 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:14.098915 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:14.098958 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:14.099179 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:14.099179 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:14.099180 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:14.099271 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:14.099317 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:14.099523 (+ 206us) server_negotiation.cc:299] Negotiation successful
0925 07:29:14.099580 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260925 07:29:14.099786 22973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:14.092771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:39809 (local address 127.5.158.65:38151)
0925 07:29:14.092915 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:14.092951 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:14.093367 (+ 416us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:14.093559 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:14.093562 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:14.094045 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:29:14.094203 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:14.094207 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:14.094786 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:14.094787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:14.095464 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:14.095466 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:14.095771 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:14.095772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:14.095853 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:14.096188 (+ 335us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:29:14.096194 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:29:14.097321 (+ 1127us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:29:14.098687 (+ 1366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:29:14.098691 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:29:14.098693 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:29:14.098859 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:29:14.099036 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:29:14.099037 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:29:14.099037 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:29:14.099126 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:29:14.099323 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:29:14.099326 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:29:14.099424 (+ 98us) client_negotiation.cc:769] Sending connection context
0925 07:29:14.099498 (+ 74us) client_negotiation.cc:240] Negotiation successful
0925 07:29:14.099598 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":86,"threads_started":1}
I20260925 07:29:14.099884 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 22842
I20260925 07:29:14.099990 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/ts-0/wal/instance
I20260925 07:29:14.100167 5753 external_mini_cluster.cc:1428] Setting key 277c861d89772ad4d1ed3c3a1dd63c43
I20260925 07:29:14.100461 22971 heartbeater.cc:347] Connected to a master server at 127.5.158.126:39809
I20260925 07:29:14.100566 22971 heartbeater.cc:464] Registering TS with master...
I20260925 07:29:14.100719 22971 heartbeater.cc:511] Master 127.5.158.126:39809 requested a full tablet report, sending...
I20260925 07:29:14.101229 22780 ts_manager.cc:194] Registered new tserver with Master: 2a45993f77334eda9966ba3763ff91a4 (127.5.158.65:43925)
I20260925 07:29:14.101776 22780 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.65@KRBTEST.COM'} at 127.5.158.65:38151
WARNING: no policy specified for kudu/127.5.158.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab.
Entry for principal kudu/127.5.158.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab.
I20260925 07:29:14.120901 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.66:0
--local_ip_for_outbound_sockets=127.5.158.66
--webserver_interface=127.5.158.66
--webserver_port=0
--tserver_master_addrs=127.5.158.126:39809
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab
--principal=kudu/127.5.158.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:40459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:29:14.192319 22976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:29:14.192468 22976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:29:14.192487 22976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:29:14.193502 22976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:29:14.193537 22976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:29:14.193559 22976 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.66
I20260925 07:29:14.194804 22976 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:40459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab
--principal=kudu/127.5.158.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.158.66
--webserver_port=0
--tserver_master_addrs=127.5.158.126:39809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22976
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.66
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:29:14.194999 22976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:29:14.195187 22976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:29:14.199340 22984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:14.199354 22982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:14.199607 22976 server_base.cc:1034] running on GCE node
W20260925 07:29:14.199345 22981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:14.199878 22976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:29:14.200115 22976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:29:14.201270 22976 hybrid_clock.cc:648] HybridClock initialized: now 1790321354201268 us; error 39 us; skew 500 ppm
Sep 25 07:29:14 dist-test-slave-1drq krb5kdc[22735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321354, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:29:14.203516 22976 init.cc:377] Logged in from keytab as kudu/127.5.158.66@KRBTEST.COM (short username kudu)
I20260925 07:29:14.203882 22976 webserver.cc:460] Webserver started at http://127.5.158.66:33411/ using document root <none> and password file <none>
I20260925 07:29:14.204088 22976 fs_manager.cc:359] Metadata directory not provided
I20260925 07:29:14.204147 22976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:29:14.204268 22976 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:29:14.205148 22976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/ts-1/data/instance:
uuid: "237485aca2534f6487d96dce3233aebc"
format_stamp: "Formatted at 2026-09-25 07:29:14 on dist-test-slave-1drq"
server_key: "466671d09e25827b119b0699ee691f49"
server_key_iv: "60cd7bfa8c35f5b4b0ea45876576a623"
server_key_version: "encryptionkey@0"
I20260925 07:29:14.205448 22976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/ts-1/wal/instance:
uuid: "237485aca2534f6487d96dce3233aebc"
format_stamp: "Formatted at 2026-09-25 07:29:14 on dist-test-slave-1drq"
server_key: "466671d09e25827b119b0699ee691f49"
server_key_iv: "60cd7bfa8c35f5b4b0ea45876576a623"
server_key_version: "encryptionkey@0"
I20260925 07:29:14.206527 22976 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 07:29:14.207105 22991 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:14.207254 22976 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 07:29:14.207332 22976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/ts-1/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/ts-1/wal
uuid: "237485aca2534f6487d96dce3233aebc"
format_stamp: "Formatted at 2026-09-25 07:29:14 on dist-test-slave-1drq"
server_key: "466671d09e25827b119b0699ee691f49"
server_key_iv: "60cd7bfa8c35f5b4b0ea45876576a623"
server_key_version: "encryptionkey@0"
I20260925 07:29:14.207405 22976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:29:14.234820 22976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:29:14.236861 22976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:29:14.237020 22976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:29:14.237228 22976 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:29:14.237489 22976 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:29:14.237540 22976 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:14.237574 22976 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:29:14.237601 22976 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:14.242981 22976 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.66:44287
I20260925 07:29:14.243027 23104 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.66:44287 every 8 connection(s)
I20260925 07:29:14.243309 22976 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
I20260925 07:29:14.244390 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 22976
I20260925 07:29:14.244505 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/ts-1/wal/instance
I20260925 07:29:14.244724 5753 external_mini_cluster.cc:1428] Setting key 6c4c5bfab40fa8513bb12cb3c4433563
Sep 25 07:29:14 dist-test-slave-1drq krb5kdc[22735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321354, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.66@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:29:14.252087 22820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:14.243919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:60685 (local address 127.5.158.126:39809)
0925 07:29:14.243957 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:14.243959 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:14.244123 (+ 164us) server_negotiation.cc:407] Connection header received
0925 07:29:14.244535 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:14.244537 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:14.244549 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:14.244607 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:14.245587 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:14.246068 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:14.246941 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:14.247133 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:14.247200 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:14.249425 (+ 2225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:14.249427 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:14.249427 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:29:14.249433 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:14.250795 (+ 1362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:14.251232 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:14.251233 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:14.251233 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:14.251277 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:14.251531 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:14.251531 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:14.251532 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:14.251620 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:14.251665 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:14.251973 (+ 308us) server_negotiation.cc:299] Negotiation successful
0925 07:29:14.252028 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 07:29:14.252302 23107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:14.243878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:39809 (local address 127.5.158.66:60685)
0925 07:29:14.244009 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:14.244043 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:14.244424 (+ 381us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:14.244671 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:14.244674 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:14.245291 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:29:14.245510 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:14.245514 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:14.246153 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:14.246154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:14.246852 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:14.246853 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:14.247205 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:14.247206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:14.247317 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:14.247726 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:29:14.247733 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:29:14.249307 (+ 1574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:29:14.250878 (+ 1571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:29:14.250885 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:29:14.250887 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:29:14.251159 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:29:14.251334 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:29:14.251335 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:29:14.251336 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:29:14.251467 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:29:14.251683 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:29:14.251688 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:29:14.251842 (+ 154us) client_negotiation.cc:769] Sending connection context
0925 07:29:14.251962 (+ 120us) client_negotiation.cc:240] Negotiation successful
0925 07:29:14.252113 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":73,"threads_started":1}
WARNING: no policy specified for kudu/127.5.158.67@KRBTEST.COM; defaulting to no policy
I20260925 07:29:14.252751 23105 heartbeater.cc:347] Connected to a master server at 127.5.158.126:39809
I20260925 07:29:14.252830 23105 heartbeater.cc:464] Registering TS with master...
I20260925 07:29:14.252942 23105 heartbeater.cc:511] Master 127.5.158.126:39809 requested a full tablet report, sending...
I20260925 07:29:14.253289 22780 ts_manager.cc:194] Registered new tserver with Master: 237485aca2534f6487d96dce3233aebc (127.5.158.66:44287)
I20260925 07:29:14.253724 22780 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.66@KRBTEST.COM'} at 127.5.158.66:60685
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab.
Entry for principal kudu/127.5.158.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab.
I20260925 07:29:14.267167 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.67:0
--local_ip_for_outbound_sockets=127.5.158.67
--webserver_interface=127.5.158.67
--webserver_port=0
--tserver_master_addrs=127.5.158.126:39809
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab
--principal=kudu/127.5.158.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:40459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:29:14.338564 23110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:29:14.338717 23110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:29:14.338735 23110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:29:14.339747 23110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:29:14.339778 23110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:29:14.339803 23110 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.67
I20260925 07:29:14.341048 23110 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:40459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab
--principal=kudu/127.5.158.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.158.67
--webserver_port=0
--tserver_master_addrs=127.5.158.126:39809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23110
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.67
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:29:14.341261 23110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:29:14.341415 23110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:29:14.345604 23116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:14.345636 23118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:14.345697 23110 server_base.cc:1034] running on GCE node
W20260925 07:29:14.345666 23115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:14.345973 23110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:29:14.346200 23110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:29:14.347337 23110 hybrid_clock.cc:648] HybridClock initialized: now 1790321354347329 us; error 33 us; skew 500 ppm
Sep 25 07:29:14 dist-test-slave-1drq krb5kdc[22735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321354, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:29:14.349566 23110 init.cc:377] Logged in from keytab as kudu/127.5.158.67@KRBTEST.COM (short username kudu)
I20260925 07:29:14.349973 23110 webserver.cc:460] Webserver started at http://127.5.158.67:42249/ using document root <none> and password file <none>
I20260925 07:29:14.350147 23110 fs_manager.cc:359] Metadata directory not provided
I20260925 07:29:14.350193 23110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:29:14.350283 23110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:29:14.351106 23110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/ts-2/data/instance:
uuid: "c9efb01524b54b268bd54fe687bc35e2"
format_stamp: "Formatted at 2026-09-25 07:29:14 on dist-test-slave-1drq"
server_key: "7024a8aff9296d8168ba60c1cd7d7af4"
server_key_iv: "1dbb4f9afb7a0411b980067edd61885e"
server_key_version: "encryptionkey@0"
I20260925 07:29:14.351374 23110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/ts-2/wal/instance:
uuid: "c9efb01524b54b268bd54fe687bc35e2"
format_stamp: "Formatted at 2026-09-25 07:29:14 on dist-test-slave-1drq"
server_key: "7024a8aff9296d8168ba60c1cd7d7af4"
server_key_iv: "1dbb4f9afb7a0411b980067edd61885e"
server_key_version: "encryptionkey@0"
I20260925 07:29:14.352464 23110 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 07:29:14.353025 23125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:14.353189 23110 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:14.353272 23110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/ts-2/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/ts-2/wal
uuid: "c9efb01524b54b268bd54fe687bc35e2"
format_stamp: "Formatted at 2026-09-25 07:29:14 on dist-test-slave-1drq"
server_key: "7024a8aff9296d8168ba60c1cd7d7af4"
server_key_iv: "1dbb4f9afb7a0411b980067edd61885e"
server_key_version: "encryptionkey@0"
I20260925 07:29:14.353377 23110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:29:14.375881 23110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:29:14.377424 23110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:29:14.377552 23110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:29:14.377727 23110 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:29:14.377975 23110 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:29:14.378005 23110 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:14.378022 23110 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:29:14.378062 23110 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:14.383872 23110 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.67:35407
I20260925 07:29:14.383926 23238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.67:35407 every 8 connection(s)
I20260925 07:29:14.384187 23110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
Sep 25 07:29:14 dist-test-slave-1drq krb5kdc[22735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321354, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.67@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:29:14.391172 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 23110
I20260925 07:29:14.391299 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/ts-2/wal/instance
I20260925 07:29:14.391469 5753 external_mini_cluster.cc:1428] Setting key 5a0e8285d30347ab42904aebe75750de
I20260925 07:29:14.393031 22820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:14.385049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:60771 (local address 127.5.158.126:39809)
0925 07:29:14.385147 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:14.385150 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:14.385338 (+ 188us) server_negotiation.cc:407] Connection header received
0925 07:29:14.385890 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:14.385891 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:14.385908 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:14.385974 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:14.386897 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:14.387437 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:14.388395 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:14.388611 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:14.388687 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:14.390562 (+ 1875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:14.390564 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:14.390564 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:29:14.390571 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:14.391940 (+ 1369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:14.392389 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:14.392390 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:14.392390 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:14.392434 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:14.392594 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:14.392594 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:14.392595 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:14.392684 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:14.392728 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:14.392924 (+ 196us) server_negotiation.cc:299] Negotiation successful
0925 07:29:14.392977 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260925 07:29:14.393227 23241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:14.385042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:39809 (local address 127.5.158.67:60771)
0925 07:29:14.385220 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:14.385259 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:14.385758 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:14.386041 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:14.386046 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:14.386608 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:29:14.386776 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:14.386780 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:14.387537 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:14.387538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:14.388306 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:14.388308 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:14.388695 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:14.388696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:14.388821 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:14.389244 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:29:14.389251 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:29:14.390477 (+ 1226us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:29:14.392011 (+ 1534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:29:14.392017 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:29:14.392018 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:29:14.392325 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:29:14.392472 (+ 147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:29:14.392473 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:29:14.392473 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:29:14.392550 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:29:14.392734 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:29:14.392738 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:29:14.392830 (+ 92us) client_negotiation.cc:769] Sending connection context
0925 07:29:14.392900 (+ 70us) client_negotiation.cc:240] Negotiation successful
0925 07:29:14.392996 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":108,"threads_started":1}
I20260925 07:29:14.393692 23239 heartbeater.cc:347] Connected to a master server at 127.5.158.126:39809
I20260925 07:29:14.393756 23239 heartbeater.cc:464] Registering TS with master...
I20260925 07:29:14.393863 23239 heartbeater.cc:511] Master 127.5.158.126:39809 requested a full tablet report, sending...
I20260925 07:29:14.394126 22780 ts_manager.cc:194] Registered new tserver with Master: c9efb01524b54b268bd54fe687bc35e2 (127.5.158.67:35407)
I20260925 07:29:14.394462 22780 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.67@KRBTEST.COM'} at 127.5.158.67:60771
I20260925 07:29:14.402947 5753 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 07:29:14.408979 22820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:14.404175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45530 (local address 127.5.158.126:39809)
0925 07:29:14.404229 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:14.404230 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:14.404269 (+ 39us) server_negotiation.cc:407] Connection header received
0925 07:29:14.404304 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:14.404305 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:14.404313 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:14.404352 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:14.404800 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:14.405243 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:14.405855 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:14.406030 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:14.406073 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:14.407017 (+ 944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:14.407018 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:14.407018 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:29:14.407023 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:14.408188 (+ 1165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:14.408430 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:14.408430 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:14.408431 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:14.408471 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:14.408592 (+ 121us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:14.408593 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:14.408593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:14.408699 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:14.408759 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:14.408834 (+ 75us) server_negotiation.cc:299] Negotiation successful
0925 07:29:14.408883 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260925 07:29:14.410459 22780 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45530:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 07:29:14.412182 22780 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 07:29:14.417909 23252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:14.414267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51072 (local address 127.5.158.66:44287)
0925 07:29:14.414425 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:14.414431 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:14.414442 (+ 11us) server_negotiation.cc:407] Connection header received
0925 07:29:14.414481 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:14.414483 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:14.414499 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:14.414590 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:29:14.414737 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:14.415840 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:14.417005 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:14.417494 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:14.417584 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:14.417638 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:14.417793 (+ 155us) server_negotiation.cc:299] Negotiation successful
0925 07:29:14.417832 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":78,"threads_started":1}
I20260925 07:29:14.417924 23249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:14.414062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.66:44287 (local address 127.0.0.1:51072)
0925 07:29:14.414215 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:14.414247 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:14.414300 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:14.414598 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:14.414600 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:14.414601 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:29:14.414672 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:14.414674 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:14.415937 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:14.415937 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:14.416922 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:14.416925 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:14.417588 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:14.417588 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:14.417676 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:14.417687 (+ 11us) client_negotiation.cc:769] Sending connection context
0925 07:29:14.417776 (+ 89us) client_negotiation.cc:240] Negotiation successful
0925 07:29:14.417827 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":51,"threads_started":1}
I20260925 07:29:14.418102 23247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:14.414063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.67:35407 (local address 127.0.0.1:55878)
0925 07:29:14.414329 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:14.414337 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:14.414374 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:14.414928 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:14.414929 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:14.414931 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:29:14.414965 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:14.414966 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:14.416086 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:14.416087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:14.416834 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:14.416837 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:14.417697 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:14.417698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:14.417757 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:14.417760 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 07:29:14.417805 (+ 45us) client_negotiation.cc:240] Negotiation successful
0925 07:29:14.417841 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"mutex_wait_us":2,"spinlock_wait_cycles":384,"thread_start_us":52,"threads_started":1}
I20260925 07:29:14.418094 23251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:14.414246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55878 (local address 127.5.158.67:35407)
0925 07:29:14.414743 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:14.414747 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:14.414755 (+ 8us) server_negotiation.cc:407] Connection header received
0925 07:29:14.414784 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:14.414786 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:14.414799 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:14.414866 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:29:14.415032 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:14.415988 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:14.416891 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:14.417550 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:14.417617 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:14.417665 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:14.417983 (+ 318us) server_negotiation.cc:299] Negotiation successful
0925 07:29:14.418021 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":61,"threads_started":1}
I20260925 07:29:14.418303 23250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:14.414178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56324 (local address 127.5.158.65:43925)
0925 07:29:14.414501 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:14.414505 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:14.414514 (+ 9us) server_negotiation.cc:407] Connection header received
0925 07:29:14.414549 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:14.414551 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:14.414565 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:14.414636 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:29:14.414855 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:14.415601 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:14.417031 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:14.417884 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:14.417940 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:14.417997 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:14.418178 (+ 181us) server_negotiation.cc:299] Negotiation successful
0925 07:29:14.418221 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":74,"threads_started":1}
I20260925 07:29:14.418505 23248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:14.414076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:43925 (local address 127.0.0.1:56324)
0925 07:29:14.414448 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:14.414454 (+ 6us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:14.414477 (+ 23us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:14.414642 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:14.414643 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:14.414644 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:29:14.414692 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:14.414693 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:14.415702 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:14.415703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:14.416979 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:14.416981 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:14.418028 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:14.418029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:14.418084 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:14.418087 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 07:29:14.418410 (+ 323us) client_negotiation.cc:240] Negotiation successful
0925 07:29:14.418451 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":40,"threads_started":1}
I20260925 07:29:14.418783 23173 tablet_service.cc:1467] Processing CreateTablet for tablet 06ef5ef46af642219f06871d4c3a9bcc (DEFAULT_TABLE table=test-table [id=26024167a4934707bd640f86e77f5e20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:29:14.418931 23039 tablet_service.cc:1467] Processing CreateTablet for tablet 06ef5ef46af642219f06871d4c3a9bcc (DEFAULT_TABLE table=test-table [id=26024167a4934707bd640f86e77f5e20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:29:14.419109 23173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06ef5ef46af642219f06871d4c3a9bcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:29:14.419260 23039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06ef5ef46af642219f06871d4c3a9bcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:29:14.419377 22905 tablet_service.cc:1467] Processing CreateTablet for tablet 06ef5ef46af642219f06871d4c3a9bcc (DEFAULT_TABLE table=test-table [id=26024167a4934707bd640f86e77f5e20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:29:14.419648 22905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06ef5ef46af642219f06871d4c3a9bcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:29:14.421444 23253 tablet_bootstrap.cc:492] T 06ef5ef46af642219f06871d4c3a9bcc P c9efb01524b54b268bd54fe687bc35e2: Bootstrap starting.
I20260925 07:29:14.422159 23253 tablet_bootstrap.cc:654] T 06ef5ef46af642219f06871d4c3a9bcc P c9efb01524b54b268bd54fe687bc35e2: Neither blocks nor log segments found. Creating new log.
I20260925 07:29:14.422634 23255 tablet_bootstrap.cc:492] T 06ef5ef46af642219f06871d4c3a9bcc P 2a45993f77334eda9966ba3763ff91a4: Bootstrap starting.
I20260925 07:29:14.422653 23253 log.cc:824] T 06ef5ef46af642219f06871d4c3a9bcc P c9efb01524b54b268bd54fe687bc35e2: Log is configured to *not* fsync() on all Append() calls
I20260925 07:29:14.422735 23254 tablet_bootstrap.cc:492] T 06ef5ef46af642219f06871d4c3a9bcc P 237485aca2534f6487d96dce3233aebc: Bootstrap starting.
I20260925 07:29:14.423336 23255 tablet_bootstrap.cc:654] T 06ef5ef46af642219f06871d4c3a9bcc P 2a45993f77334eda9966ba3763ff91a4: Neither blocks nor log segments found. Creating new log.
I20260925 07:29:14.423336 23254 tablet_bootstrap.cc:654] T 06ef5ef46af642219f06871d4c3a9bcc P 237485aca2534f6487d96dce3233aebc: Neither blocks nor log segments found. Creating new log.
I20260925 07:29:14.423938 23255 log.cc:824] T 06ef5ef46af642219f06871d4c3a9bcc P 2a45993f77334eda9966ba3763ff91a4: Log is configured to *not* fsync() on all Append() calls
I20260925 07:29:14.423938 23254 log.cc:824] T 06ef5ef46af642219f06871d4c3a9bcc P 237485aca2534f6487d96dce3233aebc: Log is configured to *not* fsync() on all Append() calls
I20260925 07:29:14.424614 23253 tablet_bootstrap.cc:492] T 06ef5ef46af642219f06871d4c3a9bcc P c9efb01524b54b268bd54fe687bc35e2: No bootstrap required, opened a new log
I20260925 07:29:14.424680 23253 ts_tablet_manager.cc:1397] T 06ef5ef46af642219f06871d4c3a9bcc P c9efb01524b54b268bd54fe687bc35e2: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260925 07:29:14.425438 23254 tablet_bootstrap.cc:492] T 06ef5ef46af642219f06871d4c3a9bcc P 237485aca2534f6487d96dce3233aebc: No bootstrap required, opened a new log
I20260925 07:29:14.425513 23254 ts_tablet_manager.cc:1397] T 06ef5ef46af642219f06871d4c3a9bcc P 237485aca2534f6487d96dce3233aebc: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260925 07:29:14.425592 23255 tablet_bootstrap.cc:492] T 06ef5ef46af642219f06871d4c3a9bcc P 2a45993f77334eda9966ba3763ff91a4: No bootstrap required, opened a new log
I20260925 07:29:14.425643 23255 ts_tablet_manager.cc:1397] T 06ef5ef46af642219f06871d4c3a9bcc P 2a45993f77334eda9966ba3763ff91a4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260925 07:29:14.426394 23253 raft_consensus.cc:348] T 06ef5ef46af642219f06871d4c3a9bcc P c9efb01524b54b268bd54fe687bc35e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "237485aca2534f6487d96dce3233aebc" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 44287 } } peers { permanent_uuid: "c9efb01524b54b268bd54fe687bc35e2" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 35407 } } peers { permanent_uuid: "2a45993f77334eda9966ba3763ff91a4" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 43925 } }
I20260925 07:29:14.426532 23253 raft_consensus.cc:374] T 06ef5ef46af642219f06871d4c3a9bcc P c9efb01524b54b268bd54fe687bc35e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:29:14.426561 23253 raft_consensus.cc:729] T 06ef5ef46af642219f06871d4c3a9bcc P c9efb01524b54b268bd54fe687bc35e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9efb01524b54b268bd54fe687bc35e2, State: Initialized, Role: FOLLOWER
I20260925 07:29:14.426666 23253 consensus_queue.cc:260] T 06ef5ef46af642219f06871d4c3a9bcc P c9efb01524b54b268bd54fe687bc35e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "237485aca2534f6487d96dce3233aebc" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 44287 } } peers { permanent_uuid: "c9efb01524b54b268bd54fe687bc35e2" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 35407 } } peers { permanent_uuid: "2a45993f77334eda9966ba3763ff91a4" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 43925 } }
I20260925 07:29:14.426903 23253 ts_tablet_manager.cc:1428] T 06ef5ef46af642219f06871d4c3a9bcc P c9efb01524b54b268bd54fe687bc35e2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 07:29:14.426918 23239 heartbeater.cc:503] Master 127.5.158.126:39809 was elected leader, sending a full tablet report...
I20260925 07:29:14.426949 23255 raft_consensus.cc:348] T 06ef5ef46af642219f06871d4c3a9bcc P 2a45993f77334eda9966ba3763ff91a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "237485aca2534f6487d96dce3233aebc" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 44287 } } peers { permanent_uuid: "c9efb01524b54b268bd54fe687bc35e2" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 35407 } } peers { permanent_uuid: "2a45993f77334eda9966ba3763ff91a4" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 43925 } }
I20260925 07:29:14.427084 23255 raft_consensus.cc:374] T 06ef5ef46af642219f06871d4c3a9bcc P 2a45993f77334eda9966ba3763ff91a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:29:14.427116 23255 raft_consensus.cc:729] T 06ef5ef46af642219f06871d4c3a9bcc P 2a45993f77334eda9966ba3763ff91a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a45993f77334eda9966ba3763ff91a4, State: Initialized, Role: FOLLOWER
I20260925 07:29:14.427207 23255 consensus_queue.cc:260] T 06ef5ef46af642219f06871d4c3a9bcc P 2a45993f77334eda9966ba3763ff91a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "237485aca2534f6487d96dce3233aebc" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 44287 } } peers { permanent_uuid: "c9efb01524b54b268bd54fe687bc35e2" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 35407 } } peers { permanent_uuid: "2a45993f77334eda9966ba3763ff91a4" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 43925 } }
I20260925 07:29:14.427438 23255 ts_tablet_manager.cc:1428] T 06ef5ef46af642219f06871d4c3a9bcc P 2a45993f77334eda9966ba3763ff91a4: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 07:29:14.427467 22971 heartbeater.cc:503] Master 127.5.158.126:39809 was elected leader, sending a full tablet report...
I20260925 07:29:14.427335 23254 raft_consensus.cc:348] T 06ef5ef46af642219f06871d4c3a9bcc P 237485aca2534f6487d96dce3233aebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "237485aca2534f6487d96dce3233aebc" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 44287 } } peers { permanent_uuid: "c9efb01524b54b268bd54fe687bc35e2" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 35407 } } peers { permanent_uuid: "2a45993f77334eda9966ba3763ff91a4" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 43925 } }
I20260925 07:29:14.427515 23254 raft_consensus.cc:374] T 06ef5ef46af642219f06871d4c3a9bcc P 237485aca2534f6487d96dce3233aebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:29:14.427546 23254 raft_consensus.cc:729] T 06ef5ef46af642219f06871d4c3a9bcc P 237485aca2534f6487d96dce3233aebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 237485aca2534f6487d96dce3233aebc, State: Initialized, Role: FOLLOWER
I20260925 07:29:14.427744 23254 consensus_queue.cc:260] T 06ef5ef46af642219f06871d4c3a9bcc P 237485aca2534f6487d96dce3233aebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "237485aca2534f6487d96dce3233aebc" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 44287 } } peers { permanent_uuid: "c9efb01524b54b268bd54fe687bc35e2" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 35407 } } peers { permanent_uuid: "2a45993f77334eda9966ba3763ff91a4" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 43925 } }
I20260925 07:29:14.428090 23254 ts_tablet_manager.cc:1428] T 06ef5ef46af642219f06871d4c3a9bcc P 237485aca2534f6487d96dce3233aebc: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260925 07:29:14.428109 23105 heartbeater.cc:503] Master 127.5.158.126:39809 was elected leader, sending a full tablet report...
W20260925 07:29:14.493801 23106 tablet.cc:2369] T 06ef5ef46af642219f06871d4c3a9bcc P 237485aca2534f6487d96dce3233aebc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 07:29:14.592931 22972 tablet.cc:2369] T 06ef5ef46af642219f06871d4c3a9bcc P 2a45993f77334eda9966ba3763ff91a4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 07:29:14.634817 23240 tablet.cc:2369] T 06ef5ef46af642219f06871d4c3a9bcc P c9efb01524b54b268bd54fe687bc35e2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 07:29:14.760874 23259 raft_consensus.cc:482] T 06ef5ef46af642219f06871d4c3a9bcc P c9efb01524b54b268bd54fe687bc35e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 07:29:14.760993 23259 raft_consensus.cc:504] T 06ef5ef46af642219f06871d4c3a9bcc P c9efb01524b54b268bd54fe687bc35e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "237485aca2534f6487d96dce3233aebc" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 44287 } } peers { permanent_uuid: "c9efb01524b54b268bd54fe687bc35e2" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 35407 } } peers { permanent_uuid: "2a45993f77334eda9966ba3763ff91a4" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 43925 } }
I20260925 07:29:14.761417 23259 leader_election.cc:288] T 06ef5ef46af642219f06871d4c3a9bcc P c9efb01524b54b268bd54fe687bc35e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 237485aca2534f6487d96dce3233aebc (127.5.158.66:44287), 2a45993f77334eda9966ba3763ff91a4 (127.5.158.65:43925)
I20260925 07:29:14.764449 23250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:14.761767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:36953 (local address 127.5.158.65:43925)
0925 07:29:14.761821 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:14.761823 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:14.761878 (+ 55us) server_negotiation.cc:407] Connection header received
0925 07:29:14.761968 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:14.761969 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:14.761978 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:14.762043 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:29:14.762196 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:14.762683 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:14.763813 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:14.764143 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:14.764220 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:14.764229 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:14.764354 (+ 125us) server_negotiation.cc:299] Negotiation successful
0925 07:29:14.764385 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260925 07:29:14.764459 23262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:14.761602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:43925 (local address 127.5.158.67:36953)
0925 07:29:14.761800 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:14.761811 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:14.761893 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:14.762045 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:14.762046 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:14.762048 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:29:14.762121 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:14.762122 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:14.762762 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:14.762763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:14.763725 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:14.763728 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:14.764229 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:14.764230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:14.764298 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:14.764302 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 07:29:14.764349 (+ 47us) client_negotiation.cc:240] Negotiation successful
0925 07:29:14.764385 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"mutex_wait_us":62,"thread_start_us":118,"threads_started":1}
I20260925 07:29:14.764528 23252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:14.761624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:37887 (local address 127.5.158.66:44287)
0925 07:29:14.761664 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:14.761666 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:14.761701 (+ 35us) server_negotiation.cc:407] Connection header received
0925 07:29:14.761808 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:14.761809 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:14.761821 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:14.761919 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:29:14.762122 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:14.762680 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:14.763810 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:14.764184 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:14.764271 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:14.764280 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:14.764417 (+ 137us) server_negotiation.cc:299] Negotiation successful
0925 07:29:14.764451 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 07:29:14.764535 23241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:14.761599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.66:44287 (local address 127.5.158.67:37887)
0925 07:29:14.761642 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:14.761654 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:14.761713 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:14.761926 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:14.761927 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:14.761928 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:29:14.762038 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:14.762040 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:14.762773 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:14.762774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:14.763730 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:14.763732 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:14.764270 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:14.764271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:14.764349 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:14.764352 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 07:29:14.764395 (+ 43us) client_negotiation.cc:240] Negotiation successful
0925 07:29:14.764440 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23,"mutex_wait_us":3}
I20260925 07:29:14.764861 23059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06ef5ef46af642219f06871d4c3a9bcc" candidate_uuid: "c9efb01524b54b268bd54fe687bc35e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "237485aca2534f6487d96dce3233aebc" is_pre_election: true
I20260925 07:29:14.764863 22925 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06ef5ef46af642219f06871d4c3a9bcc" candidate_uuid: "c9efb01524b54b268bd54fe687bc35e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a45993f77334eda9966ba3763ff91a4" is_pre_election: true
I20260925 07:29:14.765007 23059 raft_consensus.cc:2454] T 06ef5ef46af642219f06871d4c3a9bcc P 237485aca2534f6487d96dce3233aebc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c9efb01524b54b268bd54fe687bc35e2 in term 0.
I20260925 07:29:14.765031 22925 raft_consensus.cc:2454] T 06ef5ef46af642219f06871d4c3a9bcc P 2a45993f77334eda9966ba3763ff91a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c9efb01524b54b268bd54fe687bc35e2 in term 0.
I20260925 07:29:14.765244 23129 leader_election.cc:302] T 06ef5ef46af642219f06871d4c3a9bcc P c9efb01524b54b268bd54fe687bc35e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 237485aca2534f6487d96dce3233aebc, c9efb01524b54b268bd54fe687bc35e2; no voters:
I20260925 07:29:14.765365 23259 raft_consensus.cc:2781] T 06ef5ef46af642219f06871d4c3a9bcc P c9efb01524b54b268bd54fe687bc35e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 07:29:14.765424 23259 raft_consensus.cc:482] T 06ef5ef46af642219f06871d4c3a9bcc P c9efb01524b54b268bd54fe687bc35e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 07:29:14.765457 23259 raft_consensus.cc:3037] T 06ef5ef46af642219f06871d4c3a9bcc P c9efb01524b54b268bd54fe687bc35e2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:29:14.766165 23259 raft_consensus.cc:504] T 06ef5ef46af642219f06871d4c3a9bcc P c9efb01524b54b268bd54fe687bc35e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "237485aca2534f6487d96dce3233aebc" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 44287 } } peers { permanent_uuid: "c9efb01524b54b268bd54fe687bc35e2" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 35407 } } peers { permanent_uuid: "2a45993f77334eda9966ba3763ff91a4" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 43925 } }
I20260925 07:29:14.766320 23259 leader_election.cc:288] T 06ef5ef46af642219f06871d4c3a9bcc P c9efb01524b54b268bd54fe687bc35e2 [CANDIDATE]: Term 1 election: Requested vote from peers 237485aca2534f6487d96dce3233aebc (127.5.158.66:44287), 2a45993f77334eda9966ba3763ff91a4 (127.5.158.65:43925)
I20260925 07:29:14.766481 23059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06ef5ef46af642219f06871d4c3a9bcc" candidate_uuid: "c9efb01524b54b268bd54fe687bc35e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "237485aca2534f6487d96dce3233aebc"
I20260925 07:29:14.766515 22925 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06ef5ef46af642219f06871d4c3a9bcc" candidate_uuid: "c9efb01524b54b268bd54fe687bc35e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a45993f77334eda9966ba3763ff91a4"
I20260925 07:29:14.766578 22925 raft_consensus.cc:3037] T 06ef5ef46af642219f06871d4c3a9bcc P 2a45993f77334eda9966ba3763ff91a4 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:29:14.766578 23059 raft_consensus.cc:3037] T 06ef5ef46af642219f06871d4c3a9bcc P 237485aca2534f6487d96dce3233aebc [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:29:14.767258 23059 raft_consensus.cc:2454] T 06ef5ef46af642219f06871d4c3a9bcc P 237485aca2534f6487d96dce3233aebc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9efb01524b54b268bd54fe687bc35e2 in term 1.
I20260925 07:29:14.767283 22925 raft_consensus.cc:2454] T 06ef5ef46af642219f06871d4c3a9bcc P 2a45993f77334eda9966ba3763ff91a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9efb01524b54b268bd54fe687bc35e2 in term 1.
I20260925 07:29:14.767452 23128 leader_election.cc:302] T 06ef5ef46af642219f06871d4c3a9bcc P c9efb01524b54b268bd54fe687bc35e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 237485aca2534f6487d96dce3233aebc, 2a45993f77334eda9966ba3763ff91a4, c9efb01524b54b268bd54fe687bc35e2; no voters:
I20260925 07:29:14.767565 23259 raft_consensus.cc:2781] T 06ef5ef46af642219f06871d4c3a9bcc P c9efb01524b54b268bd54fe687bc35e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:29:14.767702 23259 raft_consensus.cc:686] T 06ef5ef46af642219f06871d4c3a9bcc P c9efb01524b54b268bd54fe687bc35e2 [term 1 LEADER]: Becoming Leader. State: Replica: c9efb01524b54b268bd54fe687bc35e2, State: Running, Role: LEADER
I20260925 07:29:14.767796 23259 consensus_queue.cc:237] T 06ef5ef46af642219f06871d4c3a9bcc P c9efb01524b54b268bd54fe687bc35e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "237485aca2534f6487d96dce3233aebc" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 44287 } } peers { permanent_uuid: "c9efb01524b54b268bd54fe687bc35e2" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 35407 } } peers { permanent_uuid: "2a45993f77334eda9966ba3763ff91a4" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 43925 } }
I20260925 07:29:14.768579 22779 catalog_manager.cc:5495] T 06ef5ef46af642219f06871d4c3a9bcc P c9efb01524b54b268bd54fe687bc35e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to c9efb01524b54b268bd54fe687bc35e2 (127.5.158.67). New cstate: current_term: 1 leader_uuid: "c9efb01524b54b268bd54fe687bc35e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "237485aca2534f6487d96dce3233aebc" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 44287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9efb01524b54b268bd54fe687bc35e2" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 35407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2a45993f77334eda9966ba3763ff91a4" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 43925 } health_report { overall_health: UNKNOWN } } }
I20260925 07:29:14.855841 23251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:14.853417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55894 (local address 127.5.158.67:35407)
0925 07:29:14.853482 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:14.853484 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:14.853494 (+ 10us) server_negotiation.cc:407] Connection header received
0925 07:29:14.853559 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:14.853560 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:14.853580 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:14.853671 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 07:29:14.853808 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:14.854278 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:14.855070 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:14.855281 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:14.855351 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:14.855500 (+ 149us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 07:29:14.855606 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 07:29:14.855705 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:14.855727 (+ 22us) server_negotiation.cc:299] Negotiation successful
0925 07:29:14.855748 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 07:29:14.857448 22925 raft_consensus.cc:1261] T 06ef5ef46af642219f06871d4c3a9bcc P 2a45993f77334eda9966ba3763ff91a4 [term 1 FOLLOWER]: Refusing update from remote peer c9efb01524b54b268bd54fe687bc35e2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:29:14.857473 23059 raft_consensus.cc:1261] T 06ef5ef46af642219f06871d4c3a9bcc P 237485aca2534f6487d96dce3233aebc [term 1 FOLLOWER]: Refusing update from remote peer c9efb01524b54b268bd54fe687bc35e2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:29:14.857782 23259 consensus_queue.cc:1035] T 06ef5ef46af642219f06871d4c3a9bcc P c9efb01524b54b268bd54fe687bc35e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a45993f77334eda9966ba3763ff91a4" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 43925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:29:14.857923 23263 consensus_queue.cc:1035] T 06ef5ef46af642219f06871d4c3a9bcc P c9efb01524b54b268bd54fe687bc35e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "237485aca2534f6487d96dce3233aebc" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 44287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:29:14.860387 23267 mvcc.cc:205] Tried to move back new op lower bound from 7333156269493788672 to 7333156269129424896. Current Snapshot: MvccSnapshot[applied={T|T < 7333156269493788672}]
I20260925 07:29:14.862006 22780 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45530:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 07:29:14.862089 22780 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45530:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 07:29:14.862727 22780 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 40a36ac07bfd4dda9396d80a11b28510: Sending DeleteTablet for 3 replicas of tablet 06ef5ef46af642219f06871d4c3a9bcc
I20260925 07:29:14.862968 23039 tablet_service.cc:1514] Processing DeleteTablet for tablet 06ef5ef46af642219f06871d4c3a9bcc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:29:14 UTC) from {username='kudu', principal='kudu/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:51072
I20260925 07:29:14.863030 22905 tablet_service.cc:1514] Processing DeleteTablet for tablet 06ef5ef46af642219f06871d4c3a9bcc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:29:14 UTC) from {username='kudu', principal='kudu/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:56324
I20260925 07:29:14.863170 23276 tablet_replica.cc:330] T 06ef5ef46af642219f06871d4c3a9bcc P 237485aca2534f6487d96dce3233aebc: stopping tablet replica
I20260925 07:29:14.863248 23277 tablet_replica.cc:330] T 06ef5ef46af642219f06871d4c3a9bcc P 2a45993f77334eda9966ba3763ff91a4: stopping tablet replica
I20260925 07:29:14.863297 23276 raft_consensus.cc:2229] T 06ef5ef46af642219f06871d4c3a9bcc P 237485aca2534f6487d96dce3233aebc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 07:29:14.863303 23173 tablet_service.cc:1514] Processing DeleteTablet for tablet 06ef5ef46af642219f06871d4c3a9bcc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:29:14 UTC) from {username='kudu', principal='kudu/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:55878
I20260925 07:29:14.863402 23277 raft_consensus.cc:2229] T 06ef5ef46af642219f06871d4c3a9bcc P 2a45993f77334eda9966ba3763ff91a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 07:29:14.863440 23276 raft_consensus.cc:2258] T 06ef5ef46af642219f06871d4c3a9bcc P 237485aca2534f6487d96dce3233aebc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:29:14.863521 23277 raft_consensus.cc:2258] T 06ef5ef46af642219f06871d4c3a9bcc P 2a45993f77334eda9966ba3763ff91a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:29:14.863664 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 22842
I20260925 07:29:14.864084 23276 ts_tablet_manager.cc:1905] T 06ef5ef46af642219f06871d4c3a9bcc P 237485aca2534f6487d96dce3233aebc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:29:14.865872 23276 ts_tablet_manager.cc:1918] T 06ef5ef46af642219f06871d4c3a9bcc P 237485aca2534f6487d96dce3233aebc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 07:29:14.865928 23276 log.cc:1196] T 06ef5ef46af642219f06871d4c3a9bcc P 237485aca2534f6487d96dce3233aebc: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/ts-1/wal/wals/06ef5ef46af642219f06871d4c3a9bcc
I20260925 07:29:14.866135 23276 ts_tablet_manager.cc:1939] T 06ef5ef46af642219f06871d4c3a9bcc P 237485aca2534f6487d96dce3233aebc: Deleting consensus metadata
I20260925 07:29:14.866768 22767 catalog_manager.cc:4841] TS 237485aca2534f6487d96dce3233aebc (127.5.158.66:44287): tablet 06ef5ef46af642219f06871d4c3a9bcc (table test-table [id=26024167a4934707bd640f86e77f5e20]) successfully deleted
I20260925 07:29:14.869218 23278 tablet_replica.cc:330] T 06ef5ef46af642219f06871d4c3a9bcc P c9efb01524b54b268bd54fe687bc35e2: stopping tablet replica
I20260925 07:29:14.869319 23278 raft_consensus.cc:2229] T 06ef5ef46af642219f06871d4c3a9bcc P c9efb01524b54b268bd54fe687bc35e2 [term 1 LEADER]: Raft consensus shutting down.
I20260925 07:29:14.869401 23278 raft_consensus.cc:2258] T 06ef5ef46af642219f06871d4c3a9bcc P c9efb01524b54b268bd54fe687bc35e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:29:14.869719 23278 ts_tablet_manager.cc:1905] T 06ef5ef46af642219f06871d4c3a9bcc P c9efb01524b54b268bd54fe687bc35e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:29:14.871116 23278 ts_tablet_manager.cc:1918] T 06ef5ef46af642219f06871d4c3a9bcc P c9efb01524b54b268bd54fe687bc35e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 07:29:14.871191 23278 log.cc:1196] T 06ef5ef46af642219f06871d4c3a9bcc P c9efb01524b54b268bd54fe687bc35e2: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790321110323796-5753-0/minicluster-data/ts-2/wal/wals/06ef5ef46af642219f06871d4c3a9bcc
I20260925 07:29:14.871438 23278 ts_tablet_manager.cc:1939] T 06ef5ef46af642219f06871d4c3a9bcc P c9efb01524b54b268bd54fe687bc35e2: Deleting consensus metadata
I20260925 07:29:14.871778 22765 catalog_manager.cc:4841] TS c9efb01524b54b268bd54fe687bc35e2 (127.5.158.67:35407): tablet 06ef5ef46af642219f06871d4c3a9bcc (table test-table [id=26024167a4934707bd640f86e77f5e20]) successfully deleted
W20260925 07:29:14.871954 22766 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.158.65:43925 (error 108)
I20260925 07:29:14.872432 23248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:14.872326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:43925 (local address 127.0.0.1:56336)
0925 07:29:14.872378 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:14.872410 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.65:43925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260925 07:29:14.872545 22766 catalog_manager.cc:4584] TS 2a45993f77334eda9966ba3763ff91a4 (127.5.158.65:43925): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 06ef5ef46af642219f06871d4c3a9bcc: Network error: Client connection negotiation failed: client connection to 127.5.158.65:43925: connect: Connection refused (error 111)
I20260925 07:29:14.873214 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 22976
I20260925 07:29:14.878512 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 23110
I20260925 07:29:14.883329 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 22748
2026-09-25T07:29:14Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2147 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-25T07:29:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T07:29:14Z Disabled control of system clock
I20260925 07:29:14.909308 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:41663
--webserver_interface=127.5.158.126
--webserver_port=0
--builtin_ntp_servers=127.5.158.84:45987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.158.126:41663
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 07:29:14.985816 23283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:29:14.985984 23283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:29:14.986016 23283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:29:14.986934 23283 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:29:14.986966 23283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:29:14.986979 23283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:29:14.986991 23283 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:29:14.987002 23283 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:29:14.988059 23283 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:45987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.158.126:41663
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:41663
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23283
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:29:14.988209 23283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:29:14.988401 23283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:29:14.992610 23291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:14.992673 23283 server_base.cc:1034] running on GCE node
W20260925 07:29:14.992612 23289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:14.992633 23288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:14.992990 23283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:29:14.993331 23283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:29:14.994498 23283 hybrid_clock.cc:648] HybridClock initialized: now 1790321354994495 us; error 36 us; skew 500 ppm
I20260925 07:29:14.995518 23283 webserver.cc:460] Webserver started at http://127.5.158.126:39789/ using document root <none> and password file <none>
I20260925 07:29:14.995746 23283 fs_manager.cc:359] Metadata directory not provided
I20260925 07:29:14.995808 23283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:29:14.995929 23283 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:29:14.996796 23283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/master-0/data/instance:
uuid: "5762728548664546a195c33eeeb0bbce"
format_stamp: "Formatted at 2026-09-25 07:29:14 on dist-test-slave-1drq"
server_key: "919ddb5d8ee5bd3c7a0d2092b24fbfa3"
server_key_iv: "990e3dec181e8eccc8dddcabfcbdaa72"
server_key_version: "encryptionkey@0"
I20260925 07:29:14.997161 23283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/master-0/wal/instance:
uuid: "5762728548664546a195c33eeeb0bbce"
format_stamp: "Formatted at 2026-09-25 07:29:14 on dist-test-slave-1drq"
server_key: "919ddb5d8ee5bd3c7a0d2092b24fbfa3"
server_key_iv: "990e3dec181e8eccc8dddcabfcbdaa72"
server_key_version: "encryptionkey@0"
I20260925 07:29:14.998334 23283 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 07:29:14.998962 23297 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:14.999163 23283 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:29:14.999257 23283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "5762728548664546a195c33eeeb0bbce"
format_stamp: "Formatted at 2026-09-25 07:29:14 on dist-test-slave-1drq"
server_key: "919ddb5d8ee5bd3c7a0d2092b24fbfa3"
server_key_iv: "990e3dec181e8eccc8dddcabfcbdaa72"
server_key_version: "encryptionkey@0"
I20260925 07:29:14.999338 23283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:29:15.006865 23283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:29:15.007115 23283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:29:15.007258 23283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:29:15.010769 23283 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.126:41663
I20260925 07:29:15.010797 23349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.126:41663 every 8 connection(s)
I20260925 07:29:15.011080 23283 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:29:15.011916 23350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:29:15.013068 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 23283
I20260925 07:29:15.013217 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/master-0/wal/instance
I20260925 07:29:15.013386 5753 external_mini_cluster.cc:1428] Setting key bbb7f177a4cf971650270ab898659589
I20260925 07:29:15.014410 23350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5762728548664546a195c33eeeb0bbce: Bootstrap starting.
I20260925 07:29:15.015084 23350 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5762728548664546a195c33eeeb0bbce: Neither blocks nor log segments found. Creating new log.
I20260925 07:29:15.015364 23350 log.cc:824] T 00000000000000000000000000000000 P 5762728548664546a195c33eeeb0bbce: Log is configured to *not* fsync() on all Append() calls
I20260925 07:29:15.015992 23350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5762728548664546a195c33eeeb0bbce: No bootstrap required, opened a new log
I20260925 07:29:15.018050 23350 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5762728548664546a195c33eeeb0bbce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5762728548664546a195c33eeeb0bbce" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 41663 } }
I20260925 07:29:15.018182 23350 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5762728548664546a195c33eeeb0bbce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:29:15.018215 23350 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5762728548664546a195c33eeeb0bbce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5762728548664546a195c33eeeb0bbce, State: Initialized, Role: FOLLOWER
I20260925 07:29:15.018329 23350 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5762728548664546a195c33eeeb0bbce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5762728548664546a195c33eeeb0bbce" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 41663 } }
I20260925 07:29:15.018404 23350 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5762728548664546a195c33eeeb0bbce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 07:29:15.018435 23350 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5762728548664546a195c33eeeb0bbce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 07:29:15.018527 23350 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5762728548664546a195c33eeeb0bbce [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:29:15.018882 23353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:15.013815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58814 (local address 127.5.158.126:41663)
0925 07:29:15.013994 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:15.014001 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:15.014032 (+ 31us) server_negotiation.cc:407] Connection header received
0925 07:29:15.014515 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:15.014522 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:15.014587 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:15.014760 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:15.014908 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:15.016092 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:15.016826 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:15.017486 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:15.017627 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:15.018190 (+ 563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:15.018193 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:15.018193 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:29:15.018368 (+ 175us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:15.018404 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:29:15.018417 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:15.018489 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:15.018541 (+ 52us) server_negotiation.cc:299] Negotiation successful
0925 07:29:15.018699 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":80,"threads_started":1}
I20260925 07:29:15.019140 23350 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5762728548664546a195c33eeeb0bbce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5762728548664546a195c33eeeb0bbce" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 41663 } }
I20260925 07:29:15.019248 23350 leader_election.cc:302] T 00000000000000000000000000000000 P 5762728548664546a195c33eeeb0bbce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5762728548664546a195c33eeeb0bbce; no voters:
I20260925 07:29:15.019395 23350 leader_election.cc:288] T 00000000000000000000000000000000 P 5762728548664546a195c33eeeb0bbce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 07:29:15.019460 23355 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5762728548664546a195c33eeeb0bbce [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:29:15.019618 23355 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5762728548664546a195c33eeeb0bbce [term 1 LEADER]: Becoming Leader. State: Replica: 5762728548664546a195c33eeeb0bbce, State: Running, Role: LEADER
I20260925 07:29:15.019619 23350 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5762728548664546a195c33eeeb0bbce [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:29:15.019724 23355 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5762728548664546a195c33eeeb0bbce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5762728548664546a195c33eeeb0bbce" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 41663 } }
I20260925 07:29:15.020642 23357 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5762728548664546a195c33eeeb0bbce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5762728548664546a195c33eeeb0bbce. Latest consensus state: current_term: 1 leader_uuid: "5762728548664546a195c33eeeb0bbce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5762728548664546a195c33eeeb0bbce" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 41663 } } }
I20260925 07:29:15.020606 23356 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5762728548664546a195c33eeeb0bbce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5762728548664546a195c33eeeb0bbce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5762728548664546a195c33eeeb0bbce" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 41663 } } }
I20260925 07:29:15.020711 23357 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5762728548664546a195c33eeeb0bbce [sys.catalog]: This master's current role is: LEADER
I20260925 07:29:15.020715 23356 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5762728548664546a195c33eeeb0bbce [sys.catalog]: This master's current role is: LEADER
I20260925 07:29:15.020929 23369 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 07:29:15.021440 23369 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 07:29:15.022787 23369 catalog_manager.cc:1347] Generated new cluster ID: 3eae25dd24484eb791fd3ed9d62e6611
I20260925 07:29:15.022866 23369 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 07:29:15.036376 23369 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 07:29:15.036906 23369 catalog_manager.cc:1504] Loading token signing keys...
I20260925 07:29:15.046763 23369 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5762728548664546a195c33eeeb0bbce: Generated new TSK 0
I20260925 07:29:15.046904 23369 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 07:29:15.051119 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.65:0
--local_ip_for_outbound_sockets=127.5.158.65
--webserver_interface=127.5.158.65
--webserver_port=0
--tserver_master_addrs=127.5.158.126:41663
--builtin_ntp_servers=127.5.158.84:45987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 07:29:15.148777 23374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:29:15.148942 23374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:29:15.148962 23374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:29:15.150144 23374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:29:15.150184 23374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:29:15.150209 23374 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.65
I20260925 07:29:15.151427 23374 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:45987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.158.65
--webserver_port=0
--tserver_master_addrs=127.5.158.126:41663
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23374
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.65
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:29:15.151641 23374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:29:15.151927 23374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:29:15.157123 23382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:15.157147 23380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:15.157325 23374 server_base.cc:1034] running on GCE node
W20260925 07:29:15.157384 23379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:15.157555 23374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:29:15.157783 23374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:29:15.158941 23374 hybrid_clock.cc:648] HybridClock initialized: now 1790321355158926 us; error 28 us; skew 500 ppm
I20260925 07:29:15.160151 23374 webserver.cc:460] Webserver started at http://127.5.158.65:37837/ using document root <none> and password file <none>
I20260925 07:29:15.160414 23374 fs_manager.cc:359] Metadata directory not provided
I20260925 07:29:15.160476 23374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:29:15.160609 23374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:29:15.161604 23374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/ts-0/data/instance:
uuid: "65f995dc58fd40609b03746d92fa52ee"
format_stamp: "Formatted at 2026-09-25 07:29:15 on dist-test-slave-1drq"
server_key: "bfa47343dd0644ddd50a48bc89406763"
server_key_iv: "b2b03373ed3f2e17147160e7e3d7ff4d"
server_key_version: "encryptionkey@0"
I20260925 07:29:15.161928 23374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/ts-0/wal/instance:
uuid: "65f995dc58fd40609b03746d92fa52ee"
format_stamp: "Formatted at 2026-09-25 07:29:15 on dist-test-slave-1drq"
server_key: "bfa47343dd0644ddd50a48bc89406763"
server_key_iv: "b2b03373ed3f2e17147160e7e3d7ff4d"
server_key_version: "encryptionkey@0"
I20260925 07:29:15.163065 23374 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260925 07:29:15.163743 23388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:15.163893 23374 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:15.163983 23374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/ts-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/ts-0/wal
uuid: "65f995dc58fd40609b03746d92fa52ee"
format_stamp: "Formatted at 2026-09-25 07:29:15 on dist-test-slave-1drq"
server_key: "bfa47343dd0644ddd50a48bc89406763"
server_key_iv: "b2b03373ed3f2e17147160e7e3d7ff4d"
server_key_version: "encryptionkey@0"
I20260925 07:29:15.164057 23374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:29:15.176273 23374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:29:15.176494 23374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:29:15.176620 23374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:29:15.176823 23374 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:29:15.177130 23374 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:29:15.177168 23374 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:15.177202 23374 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:29:15.177227 23374 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:15.182555 23374 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.65:46413
I20260925 07:29:15.182591 23501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.65:46413 every 8 connection(s)
I20260925 07:29:15.182864 23374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
I20260925 07:29:15.186035 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 23374
I20260925 07:29:15.186136 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/ts-0/wal/instance
I20260925 07:29:15.186319 5753 external_mini_cluster.cc:1428] Setting key 958e5969f72c6ef7ff206296a36a4d49
I20260925 07:29:15.187243 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.66:0
--local_ip_for_outbound_sockets=127.5.158.66
--webserver_interface=127.5.158.66
--webserver_port=0
--tserver_master_addrs=127.5.158.126:41663
--builtin_ntp_servers=127.5.158.84:45987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 07:29:15.187440 23353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:15.183391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:35461 (local address 127.5.158.126:41663)
0925 07:29:15.183427 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:15.183429 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:15.183632 (+ 203us) server_negotiation.cc:407] Connection header received
0925 07:29:15.184017 (+ 385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:15.184018 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:15.184027 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:15.184076 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:15.184688 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:15.185167 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:15.186039 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:15.186324 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:15.186393 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:15.187167 (+ 774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:15.187168 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:15.187168 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:29:15.187172 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:15.187185 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:29:15.187188 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:15.187228 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:15.187331 (+ 103us) server_negotiation.cc:299] Negotiation successful
0925 07:29:15.187357 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260925 07:29:15.187657 23504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:15.183371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:41663 (local address 127.5.158.65:35461)
0925 07:29:15.183533 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:15.183566 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:15.183949 (+ 383us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:15.184129 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:15.184131 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:15.184137 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:29:15.184626 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:15.184630 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:15.185242 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:15.185243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:15.185961 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:15.185963 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:15.186409 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:15.186410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:15.186543 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:15.187017 (+ 474us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 07:29:15.187017 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:29:15.187065 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 07:29:15.187089 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:29:15.187230 (+ 141us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:29:15.187235 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:29:15.187237 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 07:29:15.187304 (+ 67us) client_negotiation.cc:240] Negotiation successful
0925 07:29:15.187402 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":79,"threads_started":1}
I20260925 07:29:15.188097 23502 heartbeater.cc:347] Connected to a master server at 127.5.158.126:41663
I20260925 07:29:15.188202 23502 heartbeater.cc:464] Registering TS with master...
I20260925 07:29:15.188364 23502 heartbeater.cc:511] Master 127.5.158.126:41663 requested a full tablet report, sending...
I20260925 07:29:15.188843 23314 ts_manager.cc:194] Registered new tserver with Master: 65f995dc58fd40609b03746d92fa52ee (127.5.158.65:46413)
I20260925 07:29:15.189518 23314 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.158.65:35461
W20260925 07:29:15.262008 23505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:29:15.262207 23505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:29:15.262240 23505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:29:15.263289 23505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:29:15.263346 23505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:29:15.263388 23505 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.66
I20260925 07:29:15.264575 23505 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:45987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.158.66
--webserver_port=0
--tserver_master_addrs=127.5.158.126:41663
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23505
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.66
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:29:15.264807 23505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:29:15.265056 23505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:29:15.269778 23510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:15.269786 23511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:15.269838 23513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:15.269845 23505 server_base.cc:1034] running on GCE node
I20260925 07:29:15.270148 23505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:29:15.270356 23505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:29:15.271481 23505 hybrid_clock.cc:648] HybridClock initialized: now 1790321355271475 us; error 31 us; skew 500 ppm
I20260925 07:29:15.272588 23505 webserver.cc:460] Webserver started at http://127.5.158.66:46181/ using document root <none> and password file <none>
I20260925 07:29:15.272771 23505 fs_manager.cc:359] Metadata directory not provided
I20260925 07:29:15.272810 23505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:29:15.272889 23505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:29:15.273808 23505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/ts-1/data/instance:
uuid: "4ea213a1aca34e0ca3c5f76fe4186945"
format_stamp: "Formatted at 2026-09-25 07:29:15 on dist-test-slave-1drq"
server_key: "793e3845f93d38c2a086305ba484b5ec"
server_key_iv: "13bbe8c4241cf3574ca533ae8e62f64a"
server_key_version: "encryptionkey@0"
I20260925 07:29:15.274065 23505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/ts-1/wal/instance:
uuid: "4ea213a1aca34e0ca3c5f76fe4186945"
format_stamp: "Formatted at 2026-09-25 07:29:15 on dist-test-slave-1drq"
server_key: "793e3845f93d38c2a086305ba484b5ec"
server_key_iv: "13bbe8c4241cf3574ca533ae8e62f64a"
server_key_version: "encryptionkey@0"
I20260925 07:29:15.275197 23505 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 07:29:15.275792 23519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:15.275950 23505 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 07:29:15.276038 23505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/ts-1/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/ts-1/wal
uuid: "4ea213a1aca34e0ca3c5f76fe4186945"
format_stamp: "Formatted at 2026-09-25 07:29:15 on dist-test-slave-1drq"
server_key: "793e3845f93d38c2a086305ba484b5ec"
server_key_iv: "13bbe8c4241cf3574ca533ae8e62f64a"
server_key_version: "encryptionkey@0"
I20260925 07:29:15.276106 23505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:29:15.303602 23505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:29:15.303834 23505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:29:15.303968 23505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:29:15.304169 23505 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:29:15.304441 23505 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:29:15.304489 23505 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:15.304519 23505 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:29:15.304543 23505 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:15.310103 23505 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.66:39871
I20260925 07:29:15.310181 23632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.66:39871 every 8 connection(s)
I20260925 07:29:15.310453 23505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
I20260925 07:29:15.312206 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 23505
I20260925 07:29:15.312309 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/ts-1/wal/instance
I20260925 07:29:15.312502 5753 external_mini_cluster.cc:1428] Setting key 5314126fd31712e88aac1a718eae9fc6
I20260925 07:29:15.313467 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.67:0
--local_ip_for_outbound_sockets=127.5.158.67
--webserver_interface=127.5.158.67
--webserver_port=0
--tserver_master_addrs=127.5.158.126:41663
--builtin_ntp_servers=127.5.158.84:45987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 07:29:15.315693 23353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:15.311059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:52279 (local address 127.5.158.126:41663)
0925 07:29:15.311103 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:15.311105 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:15.311329 (+ 224us) server_negotiation.cc:407] Connection header received
0925 07:29:15.311691 (+ 362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:15.311692 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:15.311703 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:15.311761 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:15.312448 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:15.313067 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:15.314140 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:15.314517 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:15.314595 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:15.315379 (+ 784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:15.315381 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:15.315381 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:29:15.315388 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:15.315407 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:29:15.315413 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:15.315503 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:15.315550 (+ 47us) server_negotiation.cc:299] Negotiation successful
0925 07:29:15.315576 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 07:29:15.315796 23635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:15.311065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:41663 (local address 127.5.158.66:52279)
0925 07:29:15.311224 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:15.311251 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:15.311617 (+ 366us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:15.311803 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:15.311806 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:15.311812 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:29:15.312358 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:15.312364 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:15.313197 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:15.313198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:15.314044 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:15.314046 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:15.314613 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:15.314614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:15.314721 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:15.315235 (+ 514us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 07:29:15.315236 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:29:15.315270 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 07:29:15.315290 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:29:15.315499 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:29:15.315501 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:29:15.315503 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 07:29:15.315546 (+ 43us) client_negotiation.cc:240] Negotiation successful
0925 07:29:15.315612 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":82,"threads_started":1}
I20260925 07:29:15.316267 23633 heartbeater.cc:347] Connected to a master server at 127.5.158.126:41663
I20260925 07:29:15.316383 23633 heartbeater.cc:464] Registering TS with master...
I20260925 07:29:15.316556 23633 heartbeater.cc:511] Master 127.5.158.126:41663 requested a full tablet report, sending...
I20260925 07:29:15.316954 23314 ts_manager.cc:194] Registered new tserver with Master: 4ea213a1aca34e0ca3c5f76fe4186945 (127.5.158.66:39871)
I20260925 07:29:15.317432 23314 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.158.66:52279
W20260925 07:29:15.389144 23636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:29:15.389317 23636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:29:15.389338 23636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:29:15.390457 23636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:29:15.390494 23636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:29:15.390517 23636 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.67
I20260925 07:29:15.391757 23636 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:45987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.158.67
--webserver_port=0
--tserver_master_addrs=127.5.158.126:41663
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23636
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.67
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:29:15.391968 23636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:29:15.392263 23636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:29:15.396683 23641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:15.396767 23644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:15.396721 23642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:15.396885 23636 server_base.cc:1034] running on GCE node
I20260925 07:29:15.397078 23636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:29:15.397312 23636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:29:15.398474 23636 hybrid_clock.cc:648] HybridClock initialized: now 1790321355398464 us; error 48 us; skew 500 ppm
I20260925 07:29:15.399641 23636 webserver.cc:460] Webserver started at http://127.5.158.67:45617/ using document root <none> and password file <none>
I20260925 07:29:15.399868 23636 fs_manager.cc:359] Metadata directory not provided
I20260925 07:29:15.399936 23636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:29:15.400080 23636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:29:15.401144 23636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/ts-2/data/instance:
uuid: "9e33b00963ac42a2b3ced3e8819da23d"
format_stamp: "Formatted at 2026-09-25 07:29:15 on dist-test-slave-1drq"
server_key: "7093e56da2ddf1d783dd30d3d8959389"
server_key_iv: "0b640445981dfb05f1d772f17b2f2804"
server_key_version: "encryptionkey@0"
I20260925 07:29:15.401495 23636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/ts-2/wal/instance:
uuid: "9e33b00963ac42a2b3ced3e8819da23d"
format_stamp: "Formatted at 2026-09-25 07:29:15 on dist-test-slave-1drq"
server_key: "7093e56da2ddf1d783dd30d3d8959389"
server_key_iv: "0b640445981dfb05f1d772f17b2f2804"
server_key_version: "encryptionkey@0"
I20260925 07:29:15.402637 23636 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 07:29:15.403277 23650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:15.403462 23636 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:29:15.403550 23636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/ts-2/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/ts-2/wal
uuid: "9e33b00963ac42a2b3ced3e8819da23d"
format_stamp: "Formatted at 2026-09-25 07:29:15 on dist-test-slave-1drq"
server_key: "7093e56da2ddf1d783dd30d3d8959389"
server_key_iv: "0b640445981dfb05f1d772f17b2f2804"
server_key_version: "encryptionkey@0"
I20260925 07:29:15.403631 23636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:29:15.409730 23636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:29:15.409962 23636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:29:15.410097 23636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:29:15.410293 23636 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:29:15.410575 23636 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:29:15.410630 23636 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:15.410668 23636 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:29:15.410698 23636 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:15.416268 23636 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.67:40683
I20260925 07:29:15.416303 23763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.67:40683 every 8 connection(s)
I20260925 07:29:15.416563 23636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
I20260925 07:29:15.418423 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 23636
I20260925 07:29:15.418529 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/ts-2/wal/instance
I20260925 07:29:15.418735 5753 external_mini_cluster.cc:1428] Setting key 5ab9cf4788f7dbfda9f71af9f2bfb9a3
I20260925 07:29:15.421521 23353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:15.417208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:34367 (local address 127.5.158.126:41663)
0925 07:29:15.417249 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:15.417251 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:15.417459 (+ 208us) server_negotiation.cc:407] Connection header received
0925 07:29:15.417816 (+ 357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:15.417817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:15.417827 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:15.417875 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:15.418576 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:15.419217 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:15.420403 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:15.420579 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:15.420635 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:15.421348 (+ 713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:15.421349 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:15.421349 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:29:15.421353 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:15.421366 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:29:15.421369 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:15.421409 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:15.421451 (+ 42us) server_negotiation.cc:299] Negotiation successful
0925 07:29:15.421469 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 07:29:15.421686 23766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:15.417217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:41663 (local address 127.5.158.67:34367)
0925 07:29:15.417356 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:15.417382 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:15.417743 (+ 361us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:15.417915 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:15.417918 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:15.417924 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:29:15.418459 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:15.418464 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:15.419305 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:15.419306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:15.420337 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:15.420340 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:15.420644 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:15.420645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:15.420735 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:15.421238 (+ 503us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 07:29:15.421238 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:29:15.421261 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 07:29:15.421281 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:29:15.421409 (+ 128us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:29:15.421411 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:29:15.421412 (+ 1us) client_negotiation.cc:769] Sending connection context
0925 07:29:15.421453 (+ 41us) client_negotiation.cc:240] Negotiation successful
0925 07:29:15.421516 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":61,"threads_started":1}
I20260925 07:29:15.422103 23764 heartbeater.cc:347] Connected to a master server at 127.5.158.126:41663
I20260925 07:29:15.422195 23764 heartbeater.cc:464] Registering TS with master...
I20260925 07:29:15.422346 23764 heartbeater.cc:511] Master 127.5.158.126:41663 requested a full tablet report, sending...
I20260925 07:29:15.422639 23314 ts_manager.cc:194] Registered new tserver with Master: 9e33b00963ac42a2b3ced3e8819da23d (127.5.158.67:40683)
I20260925 07:29:15.422966 23314 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.158.67:34367
I20260925 07:29:15.430431 5753 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 07:29:15.434022 23353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:15.431647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58824 (local address 127.5.158.126:41663)
0925 07:29:15.431726 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:15.431728 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:15.431783 (+ 55us) server_negotiation.cc:407] Connection header received
0925 07:29:15.431824 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:15.431824 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:15.431832 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:15.431875 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:15.432010 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:15.432435 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:15.433127 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:15.433311 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:15.433359 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:15.433834 (+ 475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:15.433835 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:15.433835 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:29:15.433838 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:15.433849 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:29:15.433852 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:15.433895 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:15.433928 (+ 33us) server_negotiation.cc:299] Negotiation successful
0925 07:29:15.433943 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 07:29:15.435279 23314 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58824:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 07:29:15.436717 23314 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 07:29:15.442675 23776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:15.438978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46256 (local address 127.5.158.66:39871)
0925 07:29:15.439189 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:15.439193 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:15.439203 (+ 10us) server_negotiation.cc:407] Connection header received
0925 07:29:15.439243 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:15.439245 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:15.439260 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:15.439339 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:29:15.439548 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:15.440664 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:15.441840 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:15.442279 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:15.442366 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:15.442467 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:15.442561 (+ 94us) server_negotiation.cc:299] Negotiation successful
0925 07:29:15.442600 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":92,"threads_started":1}
I20260925 07:29:15.442657 23773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:15.438777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.66:39871 (local address 127.0.0.1:46256)
0925 07:29:15.439038 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:15.439075 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:15.439134 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:15.439405 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:15.439406 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:15.439408 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:29:15.439469 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:15.439471 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:15.440745 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:15.440746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:15.441767 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:15.441769 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:15.442366 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:15.442367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:15.442463 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:15.442476 (+ 13us) client_negotiation.cc:769] Sending connection context
0925 07:29:15.442535 (+ 59us) client_negotiation.cc:240] Negotiation successful
0925 07:29:15.442587 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"mutex_wait_us":2,"thread_start_us":74,"threads_started":1}
I20260925 07:29:15.442785 23774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:15.438773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:46413 (local address 127.0.0.1:50604)
0925 07:29:15.438955 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:15.439150 (+ 195us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:15.439205 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:15.439372 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:15.439374 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:15.439376 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:29:15.439439 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:15.439441 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:15.440460 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:15.440461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:15.441669 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:15.441672 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:15.442574 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:15.442575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:15.442647 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:15.442651 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 07:29:15.442678 (+ 27us) client_negotiation.cc:240] Negotiation successful
0925 07:29:15.442715 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":2,"thread_start_us":43,"threads_started":1}
I20260925 07:29:15.443176 23775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:15.438860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50604 (local address 127.5.158.65:46413)
0925 07:29:15.439205 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:15.439210 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:15.439222 (+ 12us) server_negotiation.cc:407] Connection header received
0925 07:29:15.439275 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:15.439278 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:15.439294 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:15.439368 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:29:15.439536 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:15.440365 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:15.441764 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:15.442478 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:15.442931 (+ 453us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:15.443029 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:15.443048 (+ 19us) server_negotiation.cc:299] Negotiation successful
0925 07:29:15.443076 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":85,"threads_started":1}
I20260925 07:29:15.443699 23567 tablet_service.cc:1467] Processing CreateTablet for tablet 317f6af466e44b25a1688083b7239b24 (DEFAULT_TABLE table=test-table [id=5ec991c26aba48eba7dc340899f874fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:29:15.443698 23436 tablet_service.cc:1467] Processing CreateTablet for tablet 317f6af466e44b25a1688083b7239b24 (DEFAULT_TABLE table=test-table [id=5ec991c26aba48eba7dc340899f874fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:29:15.443914 23772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:15.438819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.67:40683 (local address 127.0.0.1:55010)
0925 07:29:15.439237 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:15.439245 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:15.439281 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:15.439471 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:15.439472 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:15.439474 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:29:15.439507 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:15.439508 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:15.441852 (+ 2344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:15.441852 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:15.442891 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:15.442893 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:15.443700 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:15.443701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:15.443757 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:15.443761 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 07:29:15.443804 (+ 43us) client_negotiation.cc:240] Negotiation successful
0925 07:29:15.443846 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":90,"threads_started":1}
I20260925 07:29:15.444005 23777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:15.438996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55010 (local address 127.5.158.67:40683)
0925 07:29:15.439303 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:15.439306 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:15.439316 (+ 10us) server_negotiation.cc:407] Connection header received
0925 07:29:15.439344 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:15.439346 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:15.439357 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:15.439423 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:29:15.439577 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:15.440654 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:15.442968 (+ 2314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:15.443590 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:15.443691 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:15.443863 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:15.443886 (+ 23us) server_negotiation.cc:299] Negotiation successful
0925 07:29:15.443917 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":101,"threads_started":1}
I20260925 07:29:15.444015 23436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 317f6af466e44b25a1688083b7239b24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:29:15.444015 23567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 317f6af466e44b25a1688083b7239b24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:29:15.444499 23698 tablet_service.cc:1467] Processing CreateTablet for tablet 317f6af466e44b25a1688083b7239b24 (DEFAULT_TABLE table=test-table [id=5ec991c26aba48eba7dc340899f874fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:29:15.444711 23698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 317f6af466e44b25a1688083b7239b24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:29:15.446905 23778 tablet_bootstrap.cc:492] T 317f6af466e44b25a1688083b7239b24 P 9e33b00963ac42a2b3ced3e8819da23d: Bootstrap starting.
I20260925 07:29:15.447443 23779 tablet_bootstrap.cc:492] T 317f6af466e44b25a1688083b7239b24 P 4ea213a1aca34e0ca3c5f76fe4186945: Bootstrap starting.
I20260925 07:29:15.447508 23778 tablet_bootstrap.cc:654] T 317f6af466e44b25a1688083b7239b24 P 9e33b00963ac42a2b3ced3e8819da23d: Neither blocks nor log segments found. Creating new log.
I20260925 07:29:15.447768 23778 log.cc:824] T 317f6af466e44b25a1688083b7239b24 P 9e33b00963ac42a2b3ced3e8819da23d: Log is configured to *not* fsync() on all Append() calls
I20260925 07:29:15.448014 23779 tablet_bootstrap.cc:654] T 317f6af466e44b25a1688083b7239b24 P 4ea213a1aca34e0ca3c5f76fe4186945: Neither blocks nor log segments found. Creating new log.
I20260925 07:29:15.448316 23779 log.cc:824] T 317f6af466e44b25a1688083b7239b24 P 4ea213a1aca34e0ca3c5f76fe4186945: Log is configured to *not* fsync() on all Append() calls
I20260925 07:29:15.448385 23778 tablet_bootstrap.cc:492] T 317f6af466e44b25a1688083b7239b24 P 9e33b00963ac42a2b3ced3e8819da23d: No bootstrap required, opened a new log
I20260925 07:29:15.448460 23778 ts_tablet_manager.cc:1397] T 317f6af466e44b25a1688083b7239b24 P 9e33b00963ac42a2b3ced3e8819da23d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 07:29:15.448644 23780 tablet_bootstrap.cc:492] T 317f6af466e44b25a1688083b7239b24 P 65f995dc58fd40609b03746d92fa52ee: Bootstrap starting.
I20260925 07:29:15.449255 23780 tablet_bootstrap.cc:654] T 317f6af466e44b25a1688083b7239b24 P 65f995dc58fd40609b03746d92fa52ee: Neither blocks nor log segments found. Creating new log.
I20260925 07:29:15.449452 23779 tablet_bootstrap.cc:492] T 317f6af466e44b25a1688083b7239b24 P 4ea213a1aca34e0ca3c5f76fe4186945: No bootstrap required, opened a new log
I20260925 07:29:15.449520 23779 ts_tablet_manager.cc:1397] T 317f6af466e44b25a1688083b7239b24 P 4ea213a1aca34e0ca3c5f76fe4186945: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 07:29:15.449564 23780 log.cc:824] T 317f6af466e44b25a1688083b7239b24 P 65f995dc58fd40609b03746d92fa52ee: Log is configured to *not* fsync() on all Append() calls
I20260925 07:29:15.450281 23780 tablet_bootstrap.cc:492] T 317f6af466e44b25a1688083b7239b24 P 65f995dc58fd40609b03746d92fa52ee: No bootstrap required, opened a new log
I20260925 07:29:15.450361 23780 ts_tablet_manager.cc:1397] T 317f6af466e44b25a1688083b7239b24 P 65f995dc58fd40609b03746d92fa52ee: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260925 07:29:15.451056 23778 raft_consensus.cc:348] T 317f6af466e44b25a1688083b7239b24 P 9e33b00963ac42a2b3ced3e8819da23d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65f995dc58fd40609b03746d92fa52ee" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 46413 } } peers { permanent_uuid: "4ea213a1aca34e0ca3c5f76fe4186945" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 39871 } } peers { permanent_uuid: "9e33b00963ac42a2b3ced3e8819da23d" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 40683 } }
I20260925 07:29:15.451221 23778 raft_consensus.cc:374] T 317f6af466e44b25a1688083b7239b24 P 9e33b00963ac42a2b3ced3e8819da23d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:29:15.451256 23778 raft_consensus.cc:729] T 317f6af466e44b25a1688083b7239b24 P 9e33b00963ac42a2b3ced3e8819da23d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e33b00963ac42a2b3ced3e8819da23d, State: Initialized, Role: FOLLOWER
I20260925 07:29:15.451390 23778 consensus_queue.cc:260] T 317f6af466e44b25a1688083b7239b24 P 9e33b00963ac42a2b3ced3e8819da23d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65f995dc58fd40609b03746d92fa52ee" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 46413 } } peers { permanent_uuid: "4ea213a1aca34e0ca3c5f76fe4186945" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 39871 } } peers { permanent_uuid: "9e33b00963ac42a2b3ced3e8819da23d" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 40683 } }
I20260925 07:29:15.451619 23778 ts_tablet_manager.cc:1428] T 317f6af466e44b25a1688083b7239b24 P 9e33b00963ac42a2b3ced3e8819da23d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 07:29:15.451728 23764 heartbeater.cc:503] Master 127.5.158.126:41663 was elected leader, sending a full tablet report...
I20260925 07:29:15.451730 23779 raft_consensus.cc:348] T 317f6af466e44b25a1688083b7239b24 P 4ea213a1aca34e0ca3c5f76fe4186945 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65f995dc58fd40609b03746d92fa52ee" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 46413 } } peers { permanent_uuid: "4ea213a1aca34e0ca3c5f76fe4186945" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 39871 } } peers { permanent_uuid: "9e33b00963ac42a2b3ced3e8819da23d" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 40683 } }
I20260925 07:29:15.451889 23779 raft_consensus.cc:374] T 317f6af466e44b25a1688083b7239b24 P 4ea213a1aca34e0ca3c5f76fe4186945 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:29:15.451930 23779 raft_consensus.cc:729] T 317f6af466e44b25a1688083b7239b24 P 4ea213a1aca34e0ca3c5f76fe4186945 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ea213a1aca34e0ca3c5f76fe4186945, State: Initialized, Role: FOLLOWER
I20260925 07:29:15.452040 23779 consensus_queue.cc:260] T 317f6af466e44b25a1688083b7239b24 P 4ea213a1aca34e0ca3c5f76fe4186945 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65f995dc58fd40609b03746d92fa52ee" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 46413 } } peers { permanent_uuid: "4ea213a1aca34e0ca3c5f76fe4186945" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 39871 } } peers { permanent_uuid: "9e33b00963ac42a2b3ced3e8819da23d" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 40683 } }
I20260925 07:29:15.452322 23779 ts_tablet_manager.cc:1428] T 317f6af466e44b25a1688083b7239b24 P 4ea213a1aca34e0ca3c5f76fe4186945: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 07:29:15.452333 23633 heartbeater.cc:503] Master 127.5.158.126:41663 was elected leader, sending a full tablet report...
I20260925 07:29:15.452806 23780 raft_consensus.cc:348] T 317f6af466e44b25a1688083b7239b24 P 65f995dc58fd40609b03746d92fa52ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65f995dc58fd40609b03746d92fa52ee" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 46413 } } peers { permanent_uuid: "4ea213a1aca34e0ca3c5f76fe4186945" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 39871 } } peers { permanent_uuid: "9e33b00963ac42a2b3ced3e8819da23d" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 40683 } }
I20260925 07:29:15.452992 23780 raft_consensus.cc:374] T 317f6af466e44b25a1688083b7239b24 P 65f995dc58fd40609b03746d92fa52ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:29:15.453044 23780 raft_consensus.cc:729] T 317f6af466e44b25a1688083b7239b24 P 65f995dc58fd40609b03746d92fa52ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65f995dc58fd40609b03746d92fa52ee, State: Initialized, Role: FOLLOWER
I20260925 07:29:15.453179 23780 consensus_queue.cc:260] T 317f6af466e44b25a1688083b7239b24 P 65f995dc58fd40609b03746d92fa52ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65f995dc58fd40609b03746d92fa52ee" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 46413 } } peers { permanent_uuid: "4ea213a1aca34e0ca3c5f76fe4186945" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 39871 } } peers { permanent_uuid: "9e33b00963ac42a2b3ced3e8819da23d" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 40683 } }
I20260925 07:29:15.453428 23780 ts_tablet_manager.cc:1428] T 317f6af466e44b25a1688083b7239b24 P 65f995dc58fd40609b03746d92fa52ee: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260925 07:29:15.453459 23502 heartbeater.cc:503] Master 127.5.158.126:41663 was elected leader, sending a full tablet report...
W20260925 07:29:15.560981 23634 tablet.cc:2369] T 317f6af466e44b25a1688083b7239b24 P 4ea213a1aca34e0ca3c5f76fe4186945: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 07:29:15.661842 23785 raft_consensus.cc:482] T 317f6af466e44b25a1688083b7239b24 P 4ea213a1aca34e0ca3c5f76fe4186945 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 07:29:15.661952 23785 raft_consensus.cc:504] T 317f6af466e44b25a1688083b7239b24 P 4ea213a1aca34e0ca3c5f76fe4186945 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65f995dc58fd40609b03746d92fa52ee" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 46413 } } peers { permanent_uuid: "4ea213a1aca34e0ca3c5f76fe4186945" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 39871 } } peers { permanent_uuid: "9e33b00963ac42a2b3ced3e8819da23d" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 40683 } }
I20260925 07:29:15.662273 23785 leader_election.cc:288] T 317f6af466e44b25a1688083b7239b24 P 4ea213a1aca34e0ca3c5f76fe4186945 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 65f995dc58fd40609b03746d92fa52ee (127.5.158.65:46413), 9e33b00963ac42a2b3ced3e8819da23d (127.5.158.67:40683)
I20260925 07:29:15.665073 23775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:15.662557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:43077 (local address 127.5.158.65:46413)
0925 07:29:15.662597 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:15.662599 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:15.662748 (+ 149us) server_negotiation.cc:407] Connection header received
0925 07:29:15.662776 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:15.662777 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:15.662794 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:15.662847 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:29:15.662988 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:15.663596 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:15.664471 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:15.664774 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:15.664848 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:15.664856 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:15.664983 (+ 127us) server_negotiation.cc:299] Negotiation successful
0925 07:29:15.665019 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 07:29:15.665099 23787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:15.662457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:46413 (local address 127.5.158.66:43077)
0925 07:29:15.662667 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:15.662675 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:15.662713 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:15.662857 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:15.662857 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:15.662858 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:29:15.662908 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:15.662909 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:15.663684 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:15.663684 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:15.664389 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:15.664391 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:15.664848 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:15.664849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:15.664907 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:15.664910 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 07:29:15.664973 (+ 63us) client_negotiation.cc:240] Negotiation successful
0925 07:29:15.665007 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"mutex_wait_us":69,"thread_start_us":92,"threads_started":1}
I20260925 07:29:15.665186 23635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:15.662452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.67:40683 (local address 127.5.158.66:55845)
0925 07:29:15.662508 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:15.662519 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:15.662571 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:15.662786 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:15.662787 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:15.662788 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:29:15.662864 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:15.662865 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:15.663712 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:15.663713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:15.664386 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:15.664388 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:15.664901 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:15.664902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:15.664968 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:15.664970 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 07:29:15.665041 (+ 71us) client_negotiation.cc:240] Negotiation successful
0925 07:29:15.665098 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260925 07:29:15.665176 23777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:15.662579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:55845 (local address 127.5.158.67:40683)
0925 07:29:15.662637 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:15.662638 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:15.662647 (+ 9us) server_negotiation.cc:407] Connection header received
0925 07:29:15.662685 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:15.662685 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:15.662693 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:15.662771 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:29:15.662933 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:15.663619 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:15.664480 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:15.664808 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:15.664891 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:15.664899 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:15.665048 (+ 149us) server_negotiation.cc:299] Negotiation successful
0925 07:29:15.665110 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260925 07:29:15.665446 23456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "317f6af466e44b25a1688083b7239b24" candidate_uuid: "4ea213a1aca34e0ca3c5f76fe4186945" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65f995dc58fd40609b03746d92fa52ee" is_pre_election: true
I20260925 07:29:15.665470 23718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "317f6af466e44b25a1688083b7239b24" candidate_uuid: "4ea213a1aca34e0ca3c5f76fe4186945" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e33b00963ac42a2b3ced3e8819da23d" is_pre_election: true
I20260925 07:29:15.665607 23456 raft_consensus.cc:2454] T 317f6af466e44b25a1688083b7239b24 P 65f995dc58fd40609b03746d92fa52ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ea213a1aca34e0ca3c5f76fe4186945 in term 0.
I20260925 07:29:15.665647 23718 raft_consensus.cc:2454] T 317f6af466e44b25a1688083b7239b24 P 9e33b00963ac42a2b3ced3e8819da23d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ea213a1aca34e0ca3c5f76fe4186945 in term 0.
I20260925 07:29:15.665835 23521 leader_election.cc:302] T 317f6af466e44b25a1688083b7239b24 P 4ea213a1aca34e0ca3c5f76fe4186945 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ea213a1aca34e0ca3c5f76fe4186945, 65f995dc58fd40609b03746d92fa52ee; no voters:
I20260925 07:29:15.665978 23785 raft_consensus.cc:2781] T 317f6af466e44b25a1688083b7239b24 P 4ea213a1aca34e0ca3c5f76fe4186945 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 07:29:15.666028 23785 raft_consensus.cc:482] T 317f6af466e44b25a1688083b7239b24 P 4ea213a1aca34e0ca3c5f76fe4186945 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 07:29:15.666049 23785 raft_consensus.cc:3037] T 317f6af466e44b25a1688083b7239b24 P 4ea213a1aca34e0ca3c5f76fe4186945 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:29:15.666697 23785 raft_consensus.cc:504] T 317f6af466e44b25a1688083b7239b24 P 4ea213a1aca34e0ca3c5f76fe4186945 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65f995dc58fd40609b03746d92fa52ee" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 46413 } } peers { permanent_uuid: "4ea213a1aca34e0ca3c5f76fe4186945" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 39871 } } peers { permanent_uuid: "9e33b00963ac42a2b3ced3e8819da23d" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 40683 } }
I20260925 07:29:15.666812 23785 leader_election.cc:288] T 317f6af466e44b25a1688083b7239b24 P 4ea213a1aca34e0ca3c5f76fe4186945 [CANDIDATE]: Term 1 election: Requested vote from peers 65f995dc58fd40609b03746d92fa52ee (127.5.158.65:46413), 9e33b00963ac42a2b3ced3e8819da23d (127.5.158.67:40683)
I20260925 07:29:15.666909 23456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "317f6af466e44b25a1688083b7239b24" candidate_uuid: "4ea213a1aca34e0ca3c5f76fe4186945" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65f995dc58fd40609b03746d92fa52ee"
I20260925 07:29:15.666950 23718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "317f6af466e44b25a1688083b7239b24" candidate_uuid: "4ea213a1aca34e0ca3c5f76fe4186945" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e33b00963ac42a2b3ced3e8819da23d"
W20260925 07:29:15.667025 23765 tablet.cc:2369] T 317f6af466e44b25a1688083b7239b24 P 9e33b00963ac42a2b3ced3e8819da23d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 07:29:15.667003 23456 raft_consensus.cc:3037] T 317f6af466e44b25a1688083b7239b24 P 65f995dc58fd40609b03746d92fa52ee [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:29:15.667013 23718 raft_consensus.cc:3037] T 317f6af466e44b25a1688083b7239b24 P 9e33b00963ac42a2b3ced3e8819da23d [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:29:15.668006 23718 raft_consensus.cc:2454] T 317f6af466e44b25a1688083b7239b24 P 9e33b00963ac42a2b3ced3e8819da23d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ea213a1aca34e0ca3c5f76fe4186945 in term 1.
I20260925 07:29:15.668006 23456 raft_consensus.cc:2454] T 317f6af466e44b25a1688083b7239b24 P 65f995dc58fd40609b03746d92fa52ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ea213a1aca34e0ca3c5f76fe4186945 in term 1.
I20260925 07:29:15.668179 23521 leader_election.cc:302] T 317f6af466e44b25a1688083b7239b24 P 4ea213a1aca34e0ca3c5f76fe4186945 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ea213a1aca34e0ca3c5f76fe4186945, 65f995dc58fd40609b03746d92fa52ee; no voters:
I20260925 07:29:15.668267 23785 raft_consensus.cc:2781] T 317f6af466e44b25a1688083b7239b24 P 4ea213a1aca34e0ca3c5f76fe4186945 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:29:15.668380 23785 raft_consensus.cc:686] T 317f6af466e44b25a1688083b7239b24 P 4ea213a1aca34e0ca3c5f76fe4186945 [term 1 LEADER]: Becoming Leader. State: Replica: 4ea213a1aca34e0ca3c5f76fe4186945, State: Running, Role: LEADER
I20260925 07:29:15.668434 23785 consensus_queue.cc:237] T 317f6af466e44b25a1688083b7239b24 P 4ea213a1aca34e0ca3c5f76fe4186945 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65f995dc58fd40609b03746d92fa52ee" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 46413 } } peers { permanent_uuid: "4ea213a1aca34e0ca3c5f76fe4186945" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 39871 } } peers { permanent_uuid: "9e33b00963ac42a2b3ced3e8819da23d" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 40683 } }
I20260925 07:29:15.669183 23313 catalog_manager.cc:5495] T 317f6af466e44b25a1688083b7239b24 P 4ea213a1aca34e0ca3c5f76fe4186945 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ea213a1aca34e0ca3c5f76fe4186945 (127.5.158.66). New cstate: current_term: 1 leader_uuid: "4ea213a1aca34e0ca3c5f76fe4186945" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65f995dc58fd40609b03746d92fa52ee" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 46413 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ea213a1aca34e0ca3c5f76fe4186945" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 39871 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e33b00963ac42a2b3ced3e8819da23d" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 40683 } health_report { overall_health: UNKNOWN } } }
W20260925 07:29:15.683460 23503 tablet.cc:2369] T 317f6af466e44b25a1688083b7239b24 P 65f995dc58fd40609b03746d92fa52ee: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 07:29:15.725865 23776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:15.723901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46264 (local address 127.5.158.66:39871)
0925 07:29:15.723943 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:15.723945 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:15.723952 (+ 7us) server_negotiation.cc:407] Connection header received
0925 07:29:15.724013 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:15.724014 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:15.724028 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:15.724081 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 07:29:15.724179 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:15.724593 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:15.725225 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:15.725387 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:15.725435 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:15.725543 (+ 108us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 07:29:15.725629 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 07:29:15.725702 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:15.725753 (+ 51us) server_negotiation.cc:299] Negotiation successful
0925 07:29:15.725766 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 07:29:15.727617 23456 raft_consensus.cc:1261] T 317f6af466e44b25a1688083b7239b24 P 65f995dc58fd40609b03746d92fa52ee [term 1 FOLLOWER]: Refusing update from remote peer 4ea213a1aca34e0ca3c5f76fe4186945: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:29:15.727617 23718 raft_consensus.cc:1261] T 317f6af466e44b25a1688083b7239b24 P 9e33b00963ac42a2b3ced3e8819da23d [term 1 FOLLOWER]: Refusing update from remote peer 4ea213a1aca34e0ca3c5f76fe4186945: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:29:15.727962 23785 consensus_queue.cc:1035] T 317f6af466e44b25a1688083b7239b24 P 4ea213a1aca34e0ca3c5f76fe4186945 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e33b00963ac42a2b3ced3e8819da23d" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 40683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:29:15.728103 23788 consensus_queue.cc:1035] T 317f6af466e44b25a1688083b7239b24 P 4ea213a1aca34e0ca3c5f76fe4186945 [LEADER]: Connected to new peer: Peer: permanent_uuid: "65f995dc58fd40609b03746d92fa52ee" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 46413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:29:15.730814 23793 mvcc.cc:205] Tried to move back new op lower bound from 7333156273057615872 to 7333156272818315264. Current Snapshot: MvccSnapshot[applied={T|T < 7333156273057615872}]
I20260925 07:29:15.732079 23314 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58824:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 07:29:15.732161 23314 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58824:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 07:29:15.732831 23314 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5762728548664546a195c33eeeb0bbce: Sending DeleteTablet for 3 replicas of tablet 317f6af466e44b25a1688083b7239b24
I20260925 07:29:15.733132 23436 tablet_service.cc:1514] Processing DeleteTablet for tablet 317f6af466e44b25a1688083b7239b24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:29:15 UTC) from {username='slave'} at 127.0.0.1:50604
I20260925 07:29:15.733148 23698 tablet_service.cc:1514] Processing DeleteTablet for tablet 317f6af466e44b25a1688083b7239b24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:29:15 UTC) from {username='slave'} at 127.0.0.1:55010
I20260925 07:29:15.733355 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 23374
I20260925 07:29:15.733366 23567 tablet_service.cc:1514] Processing DeleteTablet for tablet 317f6af466e44b25a1688083b7239b24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:29:15 UTC) from {username='slave'} at 127.0.0.1:46256
I20260925 07:29:15.733407 23800 tablet_replica.cc:330] T 317f6af466e44b25a1688083b7239b24 P 65f995dc58fd40609b03746d92fa52ee: stopping tablet replica
I20260925 07:29:15.733484 23800 raft_consensus.cc:2229] T 317f6af466e44b25a1688083b7239b24 P 65f995dc58fd40609b03746d92fa52ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 07:29:15.733562 23800 raft_consensus.cc:2258] T 317f6af466e44b25a1688083b7239b24 P 65f995dc58fd40609b03746d92fa52ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:29:15.733682 23801 tablet_replica.cc:330] T 317f6af466e44b25a1688083b7239b24 P 9e33b00963ac42a2b3ced3e8819da23d: stopping tablet replica
I20260925 07:29:15.734054 23801 raft_consensus.cc:2229] T 317f6af466e44b25a1688083b7239b24 P 9e33b00963ac42a2b3ced3e8819da23d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 07:29:15.734215 23801 raft_consensus.cc:2258] T 317f6af466e44b25a1688083b7239b24 P 9e33b00963ac42a2b3ced3e8819da23d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:29:15.734261 23802 tablet_replica.cc:330] T 317f6af466e44b25a1688083b7239b24 P 4ea213a1aca34e0ca3c5f76fe4186945: stopping tablet replica
I20260925 07:29:15.734393 23802 raft_consensus.cc:2229] T 317f6af466e44b25a1688083b7239b24 P 4ea213a1aca34e0ca3c5f76fe4186945 [term 1 LEADER]: Raft consensus shutting down.
I20260925 07:29:15.734506 23802 raft_consensus.cc:2258] T 317f6af466e44b25a1688083b7239b24 P 4ea213a1aca34e0ca3c5f76fe4186945 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:29:15.734658 23801 ts_tablet_manager.cc:1905] T 317f6af466e44b25a1688083b7239b24 P 9e33b00963ac42a2b3ced3e8819da23d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:29:15.734929 23802 ts_tablet_manager.cc:1905] T 317f6af466e44b25a1688083b7239b24 P 4ea213a1aca34e0ca3c5f76fe4186945: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:29:15.736426 23801 ts_tablet_manager.cc:1918] T 317f6af466e44b25a1688083b7239b24 P 9e33b00963ac42a2b3ced3e8819da23d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 07:29:15.736543 23801 log.cc:1196] T 317f6af466e44b25a1688083b7239b24 P 9e33b00963ac42a2b3ced3e8819da23d: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/ts-2/wal/wals/317f6af466e44b25a1688083b7239b24
I20260925 07:29:15.736835 23801 ts_tablet_manager.cc:1939] T 317f6af466e44b25a1688083b7239b24 P 9e33b00963ac42a2b3ced3e8819da23d: Deleting consensus metadata
I20260925 07:29:15.736845 23802 ts_tablet_manager.cc:1918] T 317f6af466e44b25a1688083b7239b24 P 4ea213a1aca34e0ca3c5f76fe4186945: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 07:29:15.736912 23802 log.cc:1196] T 317f6af466e44b25a1688083b7239b24 P 4ea213a1aca34e0ca3c5f76fe4186945: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790321110323796-5753-0/minicluster-data/ts-1/wal/wals/317f6af466e44b25a1688083b7239b24
I20260925 07:29:15.737324 23298 catalog_manager.cc:4841] TS 9e33b00963ac42a2b3ced3e8819da23d (127.5.158.67:40683): tablet 317f6af466e44b25a1688083b7239b24 (table test-table [id=5ec991c26aba48eba7dc340899f874fa]) successfully deleted
I20260925 07:29:15.737376 23802 ts_tablet_manager.cc:1939] T 317f6af466e44b25a1688083b7239b24 P 4ea213a1aca34e0ca3c5f76fe4186945: Deleting consensus metadata
I20260925 07:29:15.737761 23301 catalog_manager.cc:4841] TS 4ea213a1aca34e0ca3c5f76fe4186945 (127.5.158.66:39871): tablet 317f6af466e44b25a1688083b7239b24 (table test-table [id=5ec991c26aba48eba7dc340899f874fa]) successfully deleted
W20260925 07:29:15.740378 23299 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.158.65:46413 (error 108)
I20260925 07:29:15.740875 23772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:15.740742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:46413 (local address 127.0.0.1:50616)
0925 07:29:15.740800 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:15.740844 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.65:46413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260925 07:29:15.740897 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 23505
W20260925 07:29:15.743161 23299 catalog_manager.cc:4584] TS 65f995dc58fd40609b03746d92fa52ee (127.5.158.65:46413): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 317f6af466e44b25a1688083b7239b24: Network error: Client connection negotiation failed: client connection to 127.5.158.65:46413: connect: Connection refused (error 111)
I20260925 07:29:15.746137 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 23636
I20260925 07:29:15.750507 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 23283
2026-09-25T07:29:15Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (865 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 07:29:15 dist-test-slave-1drq krb5kdc[23806](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 07:29:15 dist-test-slave-1drq krb5kdc[23806](info): set up 2 sockets
Sep 25 07:29:15 dist-test-slave-1drq krb5kdc[23806](info): commencing operation
krb5kdc: starting...
W20260925 07:29:16.692809 5753 mini_kdc.cc:121] Time spent starting KDC: real 0.924s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Sep 25 07:29:16 dist-test-slave-1drq krb5kdc[23806](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T07:29:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T07:29:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
I20260925 07:29:16.791953 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:43759
--webserver_interface=127.5.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:35525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.158.126:43759
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:29:16.862039 23819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:29:16.862192 23819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:29:16.862210 23819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:29:16.863116 23819 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:29:16.863145 23819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:29:16.863157 23819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:29:16.863168 23819 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:29:16.863178 23819 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:29:16.864318 23819 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:35525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.158.126:43759
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:43759
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23819
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:29:16.864465 23819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:29:16.864591 23819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:29:16.868664 23827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:16.868748 23819 server_base.cc:1034] running on GCE node
W20260925 07:29:16.868675 23824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:16.868721 23825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:16.869067 23819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:29:16.869415 23819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:29:16.870570 23819 hybrid_clock.cc:648] HybridClock initialized: now 1790321356870547 us; error 38 us; skew 500 ppm
Sep 25 07:29:16 dist-test-slave-1drq krb5kdc[23806](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321356, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:29:16.878857 23819 init.cc:377] Logged in from keytab as kudu/127.5.158.126@KRBTEST.COM (short username kudu)
I20260925 07:29:16.879205 23819 webserver.cc:460] Webserver started at http://127.5.158.126:42171/ using document root <none> and password file <none>
I20260925 07:29:16.879387 23819 fs_manager.cc:359] Metadata directory not provided
I20260925 07:29:16.879426 23819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:29:16.879504 23819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:29:16.880450 23819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/master-0/data/instance:
uuid: "f35e5529d85a4803be239e6a65315030"
format_stamp: "Formatted at 2026-09-25 07:29:16 on dist-test-slave-1drq"
server_key: "29c71cda1c9e01409c2917ba180f95bc"
server_key_iv: "74321df1b2596343b4b01bf4f1696194"
server_key_version: "encryptionkey@0"
I20260925 07:29:16.880765 23819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/master-0/wal/instance:
uuid: "f35e5529d85a4803be239e6a65315030"
format_stamp: "Formatted at 2026-09-25 07:29:16 on dist-test-slave-1drq"
server_key: "29c71cda1c9e01409c2917ba180f95bc"
server_key_iv: "74321df1b2596343b4b01bf4f1696194"
server_key_version: "encryptionkey@0"
I20260925 07:29:16.882016 23819 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 07:29:16.882635 23834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:16.882771 23819 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 07:29:16.882848 23819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "f35e5529d85a4803be239e6a65315030"
format_stamp: "Formatted at 2026-09-25 07:29:16 on dist-test-slave-1drq"
server_key: "29c71cda1c9e01409c2917ba180f95bc"
server_key_iv: "74321df1b2596343b4b01bf4f1696194"
server_key_version: "encryptionkey@0"
I20260925 07:29:16.882893 23819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:29:16.892097 23819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:29:16.893606 23819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:29:16.893697 23819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:29:16.896867 23819 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.126:43759
I20260925 07:29:16.896911 23886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.126:43759 every 8 connection(s)
I20260925 07:29:16.897239 23819 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:29:16.898030 23887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:29:16.899963 23887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f35e5529d85a4803be239e6a65315030: Bootstrap starting.
I20260925 07:29:16.900545 23887 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f35e5529d85a4803be239e6a65315030: Neither blocks nor log segments found. Creating new log.
I20260925 07:29:16.900836 23887 log.cc:824] T 00000000000000000000000000000000 P f35e5529d85a4803be239e6a65315030: Log is configured to *not* fsync() on all Append() calls
I20260925 07:29:16.901502 23887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f35e5529d85a4803be239e6a65315030: No bootstrap required, opened a new log
I20260925 07:29:16.902853 23887 raft_consensus.cc:348] T 00000000000000000000000000000000 P f35e5529d85a4803be239e6a65315030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f35e5529d85a4803be239e6a65315030" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 43759 } }
I20260925 07:29:16.902983 23887 raft_consensus.cc:374] T 00000000000000000000000000000000 P f35e5529d85a4803be239e6a65315030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:29:16.903024 23887 raft_consensus.cc:729] T 00000000000000000000000000000000 P f35e5529d85a4803be239e6a65315030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f35e5529d85a4803be239e6a65315030, State: Initialized, Role: FOLLOWER
I20260925 07:29:16.903167 23887 consensus_queue.cc:260] T 00000000000000000000000000000000 P f35e5529d85a4803be239e6a65315030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f35e5529d85a4803be239e6a65315030" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 43759 } }
I20260925 07:29:16.903249 23887 raft_consensus.cc:388] T 00000000000000000000000000000000 P f35e5529d85a4803be239e6a65315030 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 07:29:16.903285 23887 raft_consensus.cc:482] T 00000000000000000000000000000000 P f35e5529d85a4803be239e6a65315030 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 07:29:16.903331 23887 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f35e5529d85a4803be239e6a65315030 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:29:16.903776 23887 raft_consensus.cc:504] T 00000000000000000000000000000000 P f35e5529d85a4803be239e6a65315030 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f35e5529d85a4803be239e6a65315030" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 43759 } }
I20260925 07:29:16.903878 23887 leader_election.cc:302] T 00000000000000000000000000000000 P f35e5529d85a4803be239e6a65315030 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f35e5529d85a4803be239e6a65315030; no voters:
I20260925 07:29:16.904033 23887 leader_election.cc:288] T 00000000000000000000000000000000 P f35e5529d85a4803be239e6a65315030 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 07:29:16.904049 23890 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f35e5529d85a4803be239e6a65315030 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:29:16.904219 23890 raft_consensus.cc:686] T 00000000000000000000000000000000 P f35e5529d85a4803be239e6a65315030 [term 1 LEADER]: Becoming Leader. State: Replica: f35e5529d85a4803be239e6a65315030, State: Running, Role: LEADER
I20260925 07:29:16.904280 23887 sys_catalog.cc:564] T 00000000000000000000000000000000 P f35e5529d85a4803be239e6a65315030 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:29:16.904346 23890 consensus_queue.cc:237] T 00000000000000000000000000000000 P f35e5529d85a4803be239e6a65315030 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f35e5529d85a4803be239e6a65315030" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 43759 } }
I20260925 07:29:16.904711 23892 sys_catalog.cc:455] T 00000000000000000000000000000000 P f35e5529d85a4803be239e6a65315030 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f35e5529d85a4803be239e6a65315030. Latest consensus state: current_term: 1 leader_uuid: "f35e5529d85a4803be239e6a65315030" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f35e5529d85a4803be239e6a65315030" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 43759 } } }
I20260925 07:29:16.904812 23891 sys_catalog.cc:455] T 00000000000000000000000000000000 P f35e5529d85a4803be239e6a65315030 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f35e5529d85a4803be239e6a65315030" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f35e5529d85a4803be239e6a65315030" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 43759 } } }
I20260925 07:29:16.904850 23892 sys_catalog.cc:458] T 00000000000000000000000000000000 P f35e5529d85a4803be239e6a65315030 [sys.catalog]: This master's current role is: LEADER
I20260925 07:29:16.904886 23891 sys_catalog.cc:458] T 00000000000000000000000000000000 P f35e5529d85a4803be239e6a65315030 [sys.catalog]: This master's current role is: LEADER
I20260925 07:29:16.905366 23899 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 07:29:16.905709 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 23819
I20260925 07:29:16.905821 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/master-0/wal/instance
I20260925 07:29:16.905867 23899 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 07:29:16.906001 5753 external_mini_cluster.cc:1428] Setting key 03ed36f036b42b6ab6033d903225bf96
I20260925 07:29:16.907689 23899 catalog_manager.cc:1347] Generated new cluster ID: 6fba21be0cf747ba879aa1e6f38f1352
I20260925 07:29:16.907738 23899 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 07:29:16 dist-test-slave-1drq krb5kdc[23806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:29:16.916247 23909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:16.906389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46266 (local address 127.5.158.126:43759)
0925 07:29:16.906549 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:16.906557 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:16.906706 (+ 149us) server_negotiation.cc:407] Connection header received
0925 07:29:16.906913 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:16.906919 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:16.906971 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:16.907103 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:16.907663 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:16.908878 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:16.909920 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:16.910516 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:16.910829 (+ 313us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:16.912973 (+ 2144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:16.912977 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:16.912977 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:29:16.912987 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:16.914784 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:16.915177 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:16.915180 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:16.915180 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:16.915270 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:16.915463 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:16.915464 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:16.915464 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:16.915717 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:16.915847 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:16.915953 (+ 106us) server_negotiation.cc:299] Negotiation successful
0925 07:29:16.916059 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":73,"threads_started":1}
I20260925 07:29:16.922727 23899 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 07:29:16.923131 23899 catalog_manager.cc:1504] Loading token signing keys...
I20260925 07:29:16.932572 23899 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f35e5529d85a4803be239e6a65315030: Generated new TSK 0
I20260925 07:29:16.932720 23899 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.158.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab.
Entry for principal kudu/127.5.158.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab.
I20260925 07:29:16.961855 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.65:0
--local_ip_for_outbound_sockets=127.5.158.65
--webserver_interface=127.5.158.65
--webserver_port=0
--tserver_master_addrs=127.5.158.126:43759
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab
--principal=kudu/127.5.158.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:35525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:29:17.041352 23913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:29:17.041538 23913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:29:17.041560 23913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:29:17.042687 23913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:29:17.042726 23913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:29:17.042778 23913 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.65
I20260925 07:29:17.044186 23913 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:35525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab
--principal=kudu/127.5.158.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.158.65
--webserver_port=0
--tserver_master_addrs=127.5.158.126:43759
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23913
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.65
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:29:17.044404 23913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:29:17.044638 23913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 07:29:17.049463 23913 server_base.cc:1034] running on GCE node
W20260925 07:29:17.049405 23918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:17.049401 23919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:17.049433 23921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:17.049739 23913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:29:17.049957 23913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:29:17.051097 23913 hybrid_clock.cc:648] HybridClock initialized: now 1790321357051084 us; error 30 us; skew 500 ppm
Sep 25 07:29:17 dist-test-slave-1drq krb5kdc[23806](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321357, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:29:17.053336 23913 init.cc:377] Logged in from keytab as kudu/127.5.158.65@KRBTEST.COM (short username kudu)
I20260925 07:29:17.053756 23913 webserver.cc:460] Webserver started at http://127.5.158.65:38561/ using document root <none> and password file <none>
I20260925 07:29:17.053982 23913 fs_manager.cc:359] Metadata directory not provided
I20260925 07:29:17.054023 23913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:29:17.054158 23913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:29:17.055044 23913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/ts-0/data/instance:
uuid: "2612ca8043fd4172a0e1f0e73109e68d"
format_stamp: "Formatted at 2026-09-25 07:29:17 on dist-test-slave-1drq"
server_key: "c5a36a74ad27a88517395a737a434458"
server_key_iv: "62cb3243a98b8d63137da54f81bdb7f5"
server_key_version: "encryptionkey@0"
I20260925 07:29:17.055366 23913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/ts-0/wal/instance:
uuid: "2612ca8043fd4172a0e1f0e73109e68d"
format_stamp: "Formatted at 2026-09-25 07:29:17 on dist-test-slave-1drq"
server_key: "c5a36a74ad27a88517395a737a434458"
server_key_iv: "62cb3243a98b8d63137da54f81bdb7f5"
server_key_version: "encryptionkey@0"
I20260925 07:29:17.056597 23913 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 07:29:17.057265 23928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:17.057420 23913 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:29:17.057499 23913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/ts-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/ts-0/wal
uuid: "2612ca8043fd4172a0e1f0e73109e68d"
format_stamp: "Formatted at 2026-09-25 07:29:17 on dist-test-slave-1drq"
server_key: "c5a36a74ad27a88517395a737a434458"
server_key_iv: "62cb3243a98b8d63137da54f81bdb7f5"
server_key_version: "encryptionkey@0"
I20260925 07:29:17.057585 23913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:29:17.072206 23913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:29:17.073856 23913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:29:17.073994 23913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:29:17.074203 23913 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:29:17.074453 23913 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:29:17.074486 23913 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:17.074527 23913 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:29:17.074549 23913 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:17.079813 23913 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.65:42789
I20260925 07:29:17.079864 24041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.65:42789 every 8 connection(s)
I20260925 07:29:17.080125 23913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
Sep 25 07:29:17 dist-test-slave-1drq krb5kdc[23806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321357, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.65@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:29:17.086135 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 23913
I20260925 07:29:17.086302 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/ts-0/wal/instance
I20260925 07:29:17.086498 5753 external_mini_cluster.cc:1428] Setting key ef89405e870d82af3d13705950696e72
I20260925 07:29:17.088698 23909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:17.080766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:45171 (local address 127.5.158.126:43759)
0925 07:29:17.080827 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:17.080829 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:17.081032 (+ 203us) server_negotiation.cc:407] Connection header received
0925 07:29:17.081483 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:17.081483 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:17.081493 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:17.081539 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:17.082309 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:17.082748 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:17.083580 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:17.083762 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:17.083804 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:17.085614 (+ 1810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:17.085615 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:17.085615 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:29:17.085621 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:17.087292 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:17.087761 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:17.087762 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:17.087762 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:17.087808 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:17.088067 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:17.088067 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:17.088067 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:17.088154 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:17.088202 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:17.088485 (+ 283us) server_negotiation.cc:299] Negotiation successful
0925 07:29:17.088545 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 07:29:17.088865 24044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:17.080768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43759 (local address 127.5.158.65:45171)
0925 07:29:17.080919 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:17.080948 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:17.081390 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:17.081600 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:17.081603 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:17.082077 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:29:17.082240 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:17.082244 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:17.082820 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:17.082821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:17.083512 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:17.083514 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:17.083823 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:17.083824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:17.083906 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:17.084248 (+ 342us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:29:17.084254 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:29:17.085534 (+ 1280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:29:17.087361 (+ 1827us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:29:17.087367 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:29:17.087369 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:29:17.087685 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:29:17.087868 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:29:17.087869 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:29:17.087869 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:29:17.088010 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:29:17.088212 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:29:17.088218 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:29:17.088365 (+ 147us) client_negotiation.cc:769] Sending connection context
0925 07:29:17.088467 (+ 102us) client_negotiation.cc:240] Negotiation successful
0925 07:29:17.088628 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":73,"threads_started":1}
I20260925 07:29:17.089527 24042 heartbeater.cc:347] Connected to a master server at 127.5.158.126:43759
I20260925 07:29:17.089601 24042 heartbeater.cc:464] Registering TS with master...
I20260925 07:29:17.089758 24042 heartbeater.cc:511] Master 127.5.158.126:43759 requested a full tablet report, sending...
I20260925 07:29:17.090191 23851 ts_manager.cc:194] Registered new tserver with Master: 2612ca8043fd4172a0e1f0e73109e68d (127.5.158.65:42789)
I20260925 07:29:17.090724 23851 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.65@KRBTEST.COM'} at 127.5.158.65:45171
WARNING: no policy specified for kudu/127.5.158.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab.
Entry for principal kudu/127.5.158.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab.
I20260925 07:29:17.107493 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.66:0
--local_ip_for_outbound_sockets=127.5.158.66
--webserver_interface=127.5.158.66
--webserver_port=0
--tserver_master_addrs=127.5.158.126:43759
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab
--principal=kudu/127.5.158.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:35525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:29:17.180737 24047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:29:17.180894 24047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:29:17.180914 24047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:29:17.181933 24047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:29:17.181969 24047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:29:17.181993 24047 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.66
I20260925 07:29:17.183183 24047 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:35525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab
--principal=kudu/127.5.158.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.158.66
--webserver_port=0
--tserver_master_addrs=127.5.158.126:43759
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24047
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.66
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:29:17.183331 24047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:29:17.183461 24047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:29:17.188280 24053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:17.188282 24055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:17.188277 24052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:17.188362 24047 server_base.cc:1034] running on GCE node
I20260925 07:29:17.188687 24047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:29:17.188946 24047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:29:17.190096 24047 hybrid_clock.cc:648] HybridClock initialized: now 1790321357190082 us; error 28 us; skew 500 ppm
Sep 25 07:29:17 dist-test-slave-1drq krb5kdc[23806](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321357, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:29:17.192203 24047 init.cc:377] Logged in from keytab as kudu/127.5.158.66@KRBTEST.COM (short username kudu)
I20260925 07:29:17.192626 24047 webserver.cc:460] Webserver started at http://127.5.158.66:41213/ using document root <none> and password file <none>
I20260925 07:29:17.192819 24047 fs_manager.cc:359] Metadata directory not provided
I20260925 07:29:17.192855 24047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:29:17.192974 24047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:29:17.193851 24047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/ts-1/data/instance:
uuid: "a023d30cd6fa4a8196e4a5c9e17b01bb"
format_stamp: "Formatted at 2026-09-25 07:29:17 on dist-test-slave-1drq"
server_key: "c2316f5693ba2fec7646ff69d5cbf1f8"
server_key_iv: "0e8dd3aaf7bf08283b2991d04046a08d"
server_key_version: "encryptionkey@0"
I20260925 07:29:17.194149 24047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/ts-1/wal/instance:
uuid: "a023d30cd6fa4a8196e4a5c9e17b01bb"
format_stamp: "Formatted at 2026-09-25 07:29:17 on dist-test-slave-1drq"
server_key: "c2316f5693ba2fec7646ff69d5cbf1f8"
server_key_iv: "0e8dd3aaf7bf08283b2991d04046a08d"
server_key_version: "encryptionkey@0"
I20260925 07:29:17.195272 24047 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 07:29:17.195853 24062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:17.196013 24047 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:29:17.196105 24047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/ts-1/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/ts-1/wal
uuid: "a023d30cd6fa4a8196e4a5c9e17b01bb"
format_stamp: "Formatted at 2026-09-25 07:29:17 on dist-test-slave-1drq"
server_key: "c2316f5693ba2fec7646ff69d5cbf1f8"
server_key_iv: "0e8dd3aaf7bf08283b2991d04046a08d"
server_key_version: "encryptionkey@0"
I20260925 07:29:17.196164 24047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:29:17.209998 24047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:29:17.211469 24047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:29:17.211606 24047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:29:17.211791 24047 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:29:17.212029 24047 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:29:17.212061 24047 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:17.212078 24047 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:29:17.212119 24047 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:17.217125 24047 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.66:38643
I20260925 07:29:17.217183 24175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.66:38643 every 8 connection(s)
I20260925 07:29:17.217398 24047 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
I20260925 07:29:17.220875 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 24047
I20260925 07:29:17.220968 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/ts-1/wal/instance
I20260925 07:29:17.221163 5753 external_mini_cluster.cc:1428] Setting key e81b457cb99005c65c6cd543ffe1dbd2
Sep 25 07:29:17 dist-test-slave-1drq krb5kdc[23806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321357, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.66@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:29:17.225394 23909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:17.217921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:38361 (local address 127.5.158.126:43759)
0925 07:29:17.217985 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:17.217988 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:17.218128 (+ 140us) server_negotiation.cc:407] Connection header received
0925 07:29:17.218456 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:17.218457 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:17.218465 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:17.218525 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:17.219205 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:17.219700 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:17.220626 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:17.220800 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:17.220841 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:17.222806 (+ 1965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:17.222807 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:17.222807 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:29:17.222814 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:17.224198 (+ 1384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:17.224524 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:17.224525 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:17.224525 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:17.224579 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:17.224780 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:17.224781 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:17.224781 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:17.224907 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:17.224973 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:17.225232 (+ 259us) server_negotiation.cc:299] Negotiation successful
0925 07:29:17.225309 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 07:29:17.225406 24178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:17.217900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43759 (local address 127.5.158.66:38361)
0925 07:29:17.218036 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:17.218063 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:17.218388 (+ 325us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:17.218565 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:17.218567 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:17.219000 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:29:17.219142 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:17.219145 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:17.219775 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:17.219776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:17.220563 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:17.220565 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:17.220869 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:17.220870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:17.221013 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:17.221446 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:29:17.221454 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:29:17.222721 (+ 1267us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:29:17.224279 (+ 1558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:29:17.224283 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:29:17.224285 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:29:17.224460 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:29:17.224638 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:29:17.224639 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:29:17.224639 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:29:17.224716 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:29:17.224972 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:29:17.224984 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:29:17.225072 (+ 88us) client_negotiation.cc:769] Sending connection context
0925 07:29:17.225198 (+ 126us) client_negotiation.cc:240] Negotiation successful
0925 07:29:17.225290 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":80,"threads_started":1}
I20260925 07:29:17.225807 24176 heartbeater.cc:347] Connected to a master server at 127.5.158.126:43759
I20260925 07:29:17.225893 24176 heartbeater.cc:464] Registering TS with master...
I20260925 07:29:17.226078 24176 heartbeater.cc:511] Master 127.5.158.126:43759 requested a full tablet report, sending...
I20260925 07:29:17.226436 23851 ts_manager.cc:194] Registered new tserver with Master: a023d30cd6fa4a8196e4a5c9e17b01bb (127.5.158.66:38643)
I20260925 07:29:17.226801 23851 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.66@KRBTEST.COM'} at 127.5.158.66:38361
WARNING: no policy specified for kudu/127.5.158.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab.
Entry for principal kudu/127.5.158.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab.
I20260925 07:29:17.243299 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.67:0
--local_ip_for_outbound_sockets=127.5.158.67
--webserver_interface=127.5.158.67
--webserver_port=0
--tserver_master_addrs=127.5.158.126:43759
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab
--principal=kudu/127.5.158.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:35525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:29:17.315995 24181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:29:17.316205 24181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:29:17.316242 24181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:29:17.317343 24181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:29:17.317396 24181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:29:17.317436 24181 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.67
I20260925 07:29:17.318825 24181 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:35525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab
--principal=kudu/127.5.158.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.158.67
--webserver_port=0
--tserver_master_addrs=127.5.158.126:43759
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24181
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.67
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:29:17.319077 24181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:29:17.319290 24181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:29:17.323770 24189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:17.323787 24187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:17.323812 24186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:17.324059 24181 server_base.cc:1034] running on GCE node
I20260925 07:29:17.324260 24181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:29:17.324438 24181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:29:17.325587 24181 hybrid_clock.cc:648] HybridClock initialized: now 1790321357325567 us; error 27 us; skew 500 ppm
Sep 25 07:29:17 dist-test-slave-1drq krb5kdc[23806](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321357, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:29:17.327742 24181 init.cc:377] Logged in from keytab as kudu/127.5.158.67@KRBTEST.COM (short username kudu)
I20260925 07:29:17.328164 24181 webserver.cc:460] Webserver started at http://127.5.158.67:32981/ using document root <none> and password file <none>
I20260925 07:29:17.328378 24181 fs_manager.cc:359] Metadata directory not provided
I20260925 07:29:17.328440 24181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:29:17.328578 24181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:29:17.329569 24181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/ts-2/data/instance:
uuid: "ad4cffebbedb4fd8bb14333a335ed442"
format_stamp: "Formatted at 2026-09-25 07:29:17 on dist-test-slave-1drq"
server_key: "68919d50001cd10731ce7cb12ef41603"
server_key_iv: "121a5a597c77df0b42ffce9b4a8acb9d"
server_key_version: "encryptionkey@0"
I20260925 07:29:17.329902 24181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/ts-2/wal/instance:
uuid: "ad4cffebbedb4fd8bb14333a335ed442"
format_stamp: "Formatted at 2026-09-25 07:29:17 on dist-test-slave-1drq"
server_key: "68919d50001cd10731ce7cb12ef41603"
server_key_iv: "121a5a597c77df0b42ffce9b4a8acb9d"
server_key_version: "encryptionkey@0"
I20260925 07:29:17.331343 24181 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 07:29:17.332034 24196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:17.332219 24181 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:29:17.332309 24181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/ts-2/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/ts-2/wal
uuid: "ad4cffebbedb4fd8bb14333a335ed442"
format_stamp: "Formatted at 2026-09-25 07:29:17 on dist-test-slave-1drq"
server_key: "68919d50001cd10731ce7cb12ef41603"
server_key_iv: "121a5a597c77df0b42ffce9b4a8acb9d"
server_key_version: "encryptionkey@0"
I20260925 07:29:17.332388 24181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:29:17.356099 24181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:29:17.357924 24181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:29:17.358068 24181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:29:17.358296 24181 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:29:17.358590 24181 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:29:17.358644 24181 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:17.358685 24181 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:29:17.358713 24181 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:17.364431 24181 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.67:41669
I20260925 07:29:17.364475 24309 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.67:41669 every 8 connection(s)
I20260925 07:29:17.364761 24181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
I20260925 07:29:17.367165 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 24181
I20260925 07:29:17.367288 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/ts-2/wal/instance
I20260925 07:29:17.367458 5753 external_mini_cluster.cc:1428] Setting key 42bbb77a2a36fb2d1be4569b04de3c29
Sep 25 07:29:17 dist-test-slave-1drq krb5kdc[23806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321357, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.67@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:29:17.373309 23909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:17.365415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:37035 (local address 127.5.158.126:43759)
0925 07:29:17.365459 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:17.365461 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:17.365641 (+ 180us) server_negotiation.cc:407] Connection header received
0925 07:29:17.366045 (+ 404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:17.366047 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:17.366057 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:17.366110 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:17.367191 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:17.367960 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:17.368974 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:17.369218 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:17.369296 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:17.371115 (+ 1819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:17.371116 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:17.371116 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:29:17.371122 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:17.372164 (+ 1042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:17.372565 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:17.372566 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:17.372566 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:17.372614 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:17.372825 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:17.372825 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:17.372825 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:17.372909 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:17.372952 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:17.373201 (+ 249us) server_negotiation.cc:299] Negotiation successful
0925 07:29:17.373255 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 07:29:17.373420 24312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:17.365337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:43759 (local address 127.5.158.67:37035)
0925 07:29:17.365477 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:17.365513 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:17.365958 (+ 445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:17.366232 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:17.366235 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:17.366888 (+ 653us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:29:17.367076 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:17.367081 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:17.368086 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:17.368086 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:17.368899 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:17.368901 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:17.369301 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:17.369302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:17.369391 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:17.369764 (+ 373us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:29:17.369770 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:29:17.371040 (+ 1270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:29:17.372226 (+ 1186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:29:17.372231 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:29:17.372233 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:29:17.372422 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:29:17.372679 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:29:17.372679 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:29:17.372680 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:29:17.372759 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:29:17.372957 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:29:17.372960 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:29:17.373052 (+ 92us) client_negotiation.cc:769] Sending connection context
0925 07:29:17.373168 (+ 116us) client_negotiation.cc:240] Negotiation successful
0925 07:29:17.373266 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":79,"threads_started":1}
I20260925 07:29:17.373849 24310 heartbeater.cc:347] Connected to a master server at 127.5.158.126:43759
I20260925 07:29:17.373934 24310 heartbeater.cc:464] Registering TS with master...
I20260925 07:29:17.374073 24310 heartbeater.cc:511] Master 127.5.158.126:43759 requested a full tablet report, sending...
I20260925 07:29:17.374405 23851 ts_manager.cc:194] Registered new tserver with Master: ad4cffebbedb4fd8bb14333a335ed442 (127.5.158.67:41669)
I20260925 07:29:17.374716 23851 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.67@KRBTEST.COM'} at 127.5.158.67:37035
I20260925 07:29:17.379407 5753 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 07:29:17.386669 23909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:17.380797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46280 (local address 127.5.158.126:43759)
0925 07:29:17.380857 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:17.380859 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:17.380960 (+ 101us) server_negotiation.cc:407] Connection header received
0925 07:29:17.381013 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:17.381014 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:17.381021 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:17.381070 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:17.381611 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:17.381994 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:17.382792 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:17.382955 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:17.383001 (+ 46us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:17.384144 (+ 1143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:17.384145 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:17.384145 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:29:17.384150 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:17.385430 (+ 1280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:17.385825 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:17.385825 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:17.385825 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:17.385871 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:17.386094 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:17.386095 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:17.386095 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:17.386204 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:17.386278 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:17.386454 (+ 176us) server_negotiation.cc:299] Negotiation successful
0925 07:29:17.386569 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260925 07:29:17.388111 23851 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46280:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 07:29:17.389660 23851 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 07:29:17.395483 24321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:17.391774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:42789 (local address 127.0.0.1:51086)
0925 07:29:17.391989 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:17.391997 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:17.392056 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:17.392494 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:17.392495 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:17.392555 (+ 60us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:29:17.392592 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:17.392593 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:17.393611 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:17.393611 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:17.394289 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:17.394291 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:17.395123 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:17.395123 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:17.395215 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:17.395229 (+ 14us) client_negotiation.cc:769] Sending connection context
0925 07:29:17.395308 (+ 79us) client_negotiation.cc:240] Negotiation successful
0925 07:29:17.395405 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":83,"threads_started":1}
I20260925 07:29:17.395484 24323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:17.392007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51086 (local address 127.5.158.65:42789)
0925 07:29:17.392223 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:17.392228 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:17.392262 (+ 34us) server_negotiation.cc:407] Connection header received
0925 07:29:17.392303 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:17.392305 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:17.392323 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:17.392402 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:29:17.392673 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:17.393516 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:17.394396 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:17.395024 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:17.395099 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:17.395145 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:17.395357 (+ 212us) server_negotiation.cc:299] Negotiation successful
0925 07:29:17.395394 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":106,"threads_started":1}
I20260925 07:29:17.395804 24319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:17.391608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.67:41669 (local address 127.0.0.1:39024)
0925 07:29:17.391781 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:17.391835 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:17.391906 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:17.392143 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:17.392145 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:17.392147 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:29:17.392212 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:17.392214 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:17.393315 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:17.393316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:17.394649 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:17.394651 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:17.395547 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:17.395548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:17.395628 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:17.395632 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 07:29:17.395685 (+ 53us) client_negotiation.cc:240] Negotiation successful
0925 07:29:17.395728 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":26,"thread_start_us":47,"threads_started":1}
I20260925 07:29:17.395956 24322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:17.391760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56842 (local address 127.5.158.66:38643)
0925 07:29:17.392182 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:17.392187 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:17.392196 (+ 9us) server_negotiation.cc:407] Connection header received
0925 07:29:17.392232 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:17.392233 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:17.392248 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:17.392323 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:29:17.392401 (+ 78us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:17.393259 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:17.394778 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:17.395484 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:17.395564 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:17.395621 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:17.395816 (+ 195us) server_negotiation.cc:299] Negotiation successful
0925 07:29:17.395871 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":103,"threads_started":1}
I20260925 07:29:17.395927 24318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:17.391673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.66:38643 (local address 127.0.0.1:56842)
0925 07:29:17.391939 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:17.391948 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:17.391999 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:17.392319 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:17.392320 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:17.392321 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:29:17.392357 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:17.392358 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:17.393364 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:17.393365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:17.394679 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:17.394682 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:17.395658 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:17.395659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:17.395724 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:17.395729 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 07:29:17.395799 (+ 70us) client_negotiation.cc:240] Negotiation successful
0925 07:29:17.395846 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"mutex_wait_us":151,"thread_start_us":52,"threads_started":1}
I20260925 07:29:17.396162 24320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:17.391678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39024 (local address 127.5.158.67:41669)
0925 07:29:17.391850 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:17.391855 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:17.391885 (+ 30us) server_negotiation.cc:407] Connection header received
0925 07:29:17.392007 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:17.392010 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:17.392024 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:17.392128 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:29:17.392287 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:17.393217 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:17.394724 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:17.395362 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:17.395435 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:17.395510 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:17.395947 (+ 437us) server_negotiation.cc:299] Negotiation successful
0925 07:29:17.395989 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":72,"threads_started":1}
I20260925 07:29:17.396281 23976 tablet_service.cc:1467] Processing CreateTablet for tablet 1ff2cd2c8f1d401d82ae7a88e606bf27 (DEFAULT_TABLE table=test-table [id=5b2b9babecad4b6a82488b14fe079681]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:29:17.396615 23976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ff2cd2c8f1d401d82ae7a88e606bf27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:29:17.396739 24110 tablet_service.cc:1467] Processing CreateTablet for tablet 1ff2cd2c8f1d401d82ae7a88e606bf27 (DEFAULT_TABLE table=test-table [id=5b2b9babecad4b6a82488b14fe079681]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:29:17.397112 24110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ff2cd2c8f1d401d82ae7a88e606bf27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:29:17.397395 24244 tablet_service.cc:1467] Processing CreateTablet for tablet 1ff2cd2c8f1d401d82ae7a88e606bf27 (DEFAULT_TABLE table=test-table [id=5b2b9babecad4b6a82488b14fe079681]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:29:17.397601 24244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ff2cd2c8f1d401d82ae7a88e606bf27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:29:17.399528 24324 tablet_bootstrap.cc:492] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P a023d30cd6fa4a8196e4a5c9e17b01bb: Bootstrap starting.
I20260925 07:29:17.400025 24325 tablet_bootstrap.cc:492] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P ad4cffebbedb4fd8bb14333a335ed442: Bootstrap starting.
I20260925 07:29:17.400245 24324 tablet_bootstrap.cc:654] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P a023d30cd6fa4a8196e4a5c9e17b01bb: Neither blocks nor log segments found. Creating new log.
I20260925 07:29:17.400708 24324 log.cc:824] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P a023d30cd6fa4a8196e4a5c9e17b01bb: Log is configured to *not* fsync() on all Append() calls
I20260925 07:29:17.400766 24325 tablet_bootstrap.cc:654] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P ad4cffebbedb4fd8bb14333a335ed442: Neither blocks nor log segments found. Creating new log.
I20260925 07:29:17.401346 24324 tablet_bootstrap.cc:492] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P a023d30cd6fa4a8196e4a5c9e17b01bb: No bootstrap required, opened a new log
I20260925 07:29:17.401379 24325 log.cc:824] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P ad4cffebbedb4fd8bb14333a335ed442: Log is configured to *not* fsync() on all Append() calls
I20260925 07:29:17.401423 24324 ts_tablet_manager.cc:1397] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P a023d30cd6fa4a8196e4a5c9e17b01bb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 07:29:17.402120 24325 tablet_bootstrap.cc:492] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P ad4cffebbedb4fd8bb14333a335ed442: No bootstrap required, opened a new log
I20260925 07:29:17.402211 24325 ts_tablet_manager.cc:1397] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P ad4cffebbedb4fd8bb14333a335ed442: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 07:29:17.402906 24324 raft_consensus.cc:348] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P a023d30cd6fa4a8196e4a5c9e17b01bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad4cffebbedb4fd8bb14333a335ed442" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41669 } } peers { permanent_uuid: "2612ca8043fd4172a0e1f0e73109e68d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42789 } } peers { permanent_uuid: "a023d30cd6fa4a8196e4a5c9e17b01bb" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 38643 } }
I20260925 07:29:17.403069 24324 raft_consensus.cc:374] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P a023d30cd6fa4a8196e4a5c9e17b01bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:29:17.403129 24324 raft_consensus.cc:729] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P a023d30cd6fa4a8196e4a5c9e17b01bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a023d30cd6fa4a8196e4a5c9e17b01bb, State: Initialized, Role: FOLLOWER
I20260925 07:29:17.403319 24324 consensus_queue.cc:260] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P a023d30cd6fa4a8196e4a5c9e17b01bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad4cffebbedb4fd8bb14333a335ed442" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41669 } } peers { permanent_uuid: "2612ca8043fd4172a0e1f0e73109e68d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42789 } } peers { permanent_uuid: "a023d30cd6fa4a8196e4a5c9e17b01bb" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 38643 } }
I20260925 07:29:17.403522 24328 tablet_bootstrap.cc:492] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P 2612ca8043fd4172a0e1f0e73109e68d: Bootstrap starting.
I20260925 07:29:17.403709 24324 ts_tablet_manager.cc:1428] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P a023d30cd6fa4a8196e4a5c9e17b01bb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 07:29:17.403831 24176 heartbeater.cc:503] Master 127.5.158.126:43759 was elected leader, sending a full tablet report...
I20260925 07:29:17.404130 24328 tablet_bootstrap.cc:654] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P 2612ca8043fd4172a0e1f0e73109e68d: Neither blocks nor log segments found. Creating new log.
I20260925 07:29:17.404675 24328 log.cc:824] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P 2612ca8043fd4172a0e1f0e73109e68d: Log is configured to *not* fsync() on all Append() calls
I20260925 07:29:17.404659 24325 raft_consensus.cc:348] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P ad4cffebbedb4fd8bb14333a335ed442 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad4cffebbedb4fd8bb14333a335ed442" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41669 } } peers { permanent_uuid: "2612ca8043fd4172a0e1f0e73109e68d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42789 } } peers { permanent_uuid: "a023d30cd6fa4a8196e4a5c9e17b01bb" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 38643 } }
I20260925 07:29:17.404809 24325 raft_consensus.cc:374] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P ad4cffebbedb4fd8bb14333a335ed442 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:29:17.404851 24325 raft_consensus.cc:729] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P ad4cffebbedb4fd8bb14333a335ed442 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad4cffebbedb4fd8bb14333a335ed442, State: Initialized, Role: FOLLOWER
I20260925 07:29:17.404949 24325 consensus_queue.cc:260] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P ad4cffebbedb4fd8bb14333a335ed442 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad4cffebbedb4fd8bb14333a335ed442" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41669 } } peers { permanent_uuid: "2612ca8043fd4172a0e1f0e73109e68d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42789 } } peers { permanent_uuid: "a023d30cd6fa4a8196e4a5c9e17b01bb" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 38643 } }
I20260925 07:29:17.405231 24310 heartbeater.cc:503] Master 127.5.158.126:43759 was elected leader, sending a full tablet report...
I20260925 07:29:17.405267 24325 ts_tablet_manager.cc:1428] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P ad4cffebbedb4fd8bb14333a335ed442: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260925 07:29:17.405340 24328 tablet_bootstrap.cc:492] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P 2612ca8043fd4172a0e1f0e73109e68d: No bootstrap required, opened a new log
I20260925 07:29:17.405386 24328 ts_tablet_manager.cc:1397] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P 2612ca8043fd4172a0e1f0e73109e68d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 07:29:17.406740 24328 raft_consensus.cc:348] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P 2612ca8043fd4172a0e1f0e73109e68d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad4cffebbedb4fd8bb14333a335ed442" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41669 } } peers { permanent_uuid: "2612ca8043fd4172a0e1f0e73109e68d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42789 } } peers { permanent_uuid: "a023d30cd6fa4a8196e4a5c9e17b01bb" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 38643 } }
I20260925 07:29:17.406852 24328 raft_consensus.cc:374] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P 2612ca8043fd4172a0e1f0e73109e68d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:29:17.406869 24328 raft_consensus.cc:729] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P 2612ca8043fd4172a0e1f0e73109e68d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2612ca8043fd4172a0e1f0e73109e68d, State: Initialized, Role: FOLLOWER
I20260925 07:29:17.406925 24328 consensus_queue.cc:260] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P 2612ca8043fd4172a0e1f0e73109e68d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad4cffebbedb4fd8bb14333a335ed442" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41669 } } peers { permanent_uuid: "2612ca8043fd4172a0e1f0e73109e68d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42789 } } peers { permanent_uuid: "a023d30cd6fa4a8196e4a5c9e17b01bb" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 38643 } }
I20260925 07:29:17.407094 24328 ts_tablet_manager.cc:1428] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P 2612ca8043fd4172a0e1f0e73109e68d: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260925 07:29:17.407162 24042 heartbeater.cc:503] Master 127.5.158.126:43759 was elected leader, sending a full tablet report...
W20260925 07:29:17.467881 24177 tablet.cc:2369] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P a023d30cd6fa4a8196e4a5c9e17b01bb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 07:29:17.580842 24043 tablet.cc:2369] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P 2612ca8043fd4172a0e1f0e73109e68d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 07:29:17.615299 24311 tablet.cc:2369] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P ad4cffebbedb4fd8bb14333a335ed442: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 07:29:17.840466 24332 raft_consensus.cc:482] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P 2612ca8043fd4172a0e1f0e73109e68d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 07:29:17.840591 24332 raft_consensus.cc:504] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P 2612ca8043fd4172a0e1f0e73109e68d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad4cffebbedb4fd8bb14333a335ed442" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41669 } } peers { permanent_uuid: "2612ca8043fd4172a0e1f0e73109e68d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42789 } } peers { permanent_uuid: "a023d30cd6fa4a8196e4a5c9e17b01bb" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 38643 } }
I20260925 07:29:17.840906 24332 leader_election.cc:288] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P 2612ca8043fd4172a0e1f0e73109e68d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad4cffebbedb4fd8bb14333a335ed442 (127.5.158.67:41669), a023d30cd6fa4a8196e4a5c9e17b01bb (127.5.158.66:38643)
I20260925 07:29:17.843701 24322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:17.841178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:56537 (local address 127.5.158.66:38643)
0925 07:29:17.841218 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:17.841220 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:17.841305 (+ 85us) server_negotiation.cc:407] Connection header received
0925 07:29:17.841381 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:17.841382 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:17.841389 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:17.841459 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:29:17.841631 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:17.842029 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:17.842982 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:17.843344 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:17.843422 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:17.843429 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:17.843590 (+ 161us) server_negotiation.cc:299] Negotiation successful
0925 07:29:17.843626 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 07:29:17.843703 24320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:17.841157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:41277 (local address 127.5.158.67:41669)
0925 07:29:17.841197 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:17.841199 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:17.841300 (+ 101us) server_negotiation.cc:407] Connection header received
0925 07:29:17.841366 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:17.841367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:17.841375 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:17.841428 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:29:17.841582 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:17.841995 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:17.842985 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:17.843337 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:17.843419 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:17.843427 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:17.843576 (+ 149us) server_negotiation.cc:299] Negotiation successful
0925 07:29:17.843612 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 07:29:17.843737 24334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:17.841109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.67:41669 (local address 127.5.158.65:41277)
0925 07:29:17.841226 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:17.841235 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:17.841310 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:17.841445 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:17.841446 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:17.841448 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:29:17.841518 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:17.841520 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:17.842069 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:17.842070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:17.842884 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:17.842886 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:17.843417 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:17.843418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:17.843494 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:17.843497 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 07:29:17.843559 (+ 62us) client_negotiation.cc:240] Negotiation successful
0925 07:29:17.843601 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"mutex_wait_us":27,"thread_start_us":54,"threads_started":1}
I20260925 07:29:17.843740 24333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:17.841115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.66:38643 (local address 127.5.158.65:56537)
0925 07:29:17.841238 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:17.841254 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:17.841319 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:17.841464 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:17.841465 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:17.841467 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:29:17.841569 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:17.841570 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:17.842101 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:17.842102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:17.842883 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:17.842885 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:17.843423 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:17.843423 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:17.843503 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:17.843507 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 07:29:17.843581 (+ 74us) client_negotiation.cc:240] Negotiation successful
0925 07:29:17.843621 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"mutex_wait_us":4,"thread_start_us":49,"threads_started":1}
I20260925 07:29:17.844084 24264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ff2cd2c8f1d401d82ae7a88e606bf27" candidate_uuid: "2612ca8043fd4172a0e1f0e73109e68d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad4cffebbedb4fd8bb14333a335ed442" is_pre_election: true
I20260925 07:29:17.844241 24264 raft_consensus.cc:2454] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P ad4cffebbedb4fd8bb14333a335ed442 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2612ca8043fd4172a0e1f0e73109e68d in term 0.
I20260925 07:29:17.844278 24130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ff2cd2c8f1d401d82ae7a88e606bf27" candidate_uuid: "2612ca8043fd4172a0e1f0e73109e68d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a023d30cd6fa4a8196e4a5c9e17b01bb" is_pre_election: true
I20260925 07:29:17.844408 24130 raft_consensus.cc:2454] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P a023d30cd6fa4a8196e4a5c9e17b01bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2612ca8043fd4172a0e1f0e73109e68d in term 0.
I20260925 07:29:17.844456 23931 leader_election.cc:302] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P 2612ca8043fd4172a0e1f0e73109e68d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2612ca8043fd4172a0e1f0e73109e68d, ad4cffebbedb4fd8bb14333a335ed442; no voters:
I20260925 07:29:17.844602 24332 raft_consensus.cc:2781] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P 2612ca8043fd4172a0e1f0e73109e68d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 07:29:17.844646 24332 raft_consensus.cc:482] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P 2612ca8043fd4172a0e1f0e73109e68d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 07:29:17.844660 24332 raft_consensus.cc:3037] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P 2612ca8043fd4172a0e1f0e73109e68d [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:29:17.845335 24332 raft_consensus.cc:504] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P 2612ca8043fd4172a0e1f0e73109e68d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad4cffebbedb4fd8bb14333a335ed442" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41669 } } peers { permanent_uuid: "2612ca8043fd4172a0e1f0e73109e68d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42789 } } peers { permanent_uuid: "a023d30cd6fa4a8196e4a5c9e17b01bb" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 38643 } }
I20260925 07:29:17.845458 24332 leader_election.cc:288] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P 2612ca8043fd4172a0e1f0e73109e68d [CANDIDATE]: Term 1 election: Requested vote from peers ad4cffebbedb4fd8bb14333a335ed442 (127.5.158.67:41669), a023d30cd6fa4a8196e4a5c9e17b01bb (127.5.158.66:38643)
I20260925 07:29:17.845597 24130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ff2cd2c8f1d401d82ae7a88e606bf27" candidate_uuid: "2612ca8043fd4172a0e1f0e73109e68d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a023d30cd6fa4a8196e4a5c9e17b01bb"
I20260925 07:29:17.845602 24264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ff2cd2c8f1d401d82ae7a88e606bf27" candidate_uuid: "2612ca8043fd4172a0e1f0e73109e68d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad4cffebbedb4fd8bb14333a335ed442"
I20260925 07:29:17.845672 24130 raft_consensus.cc:3037] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P a023d30cd6fa4a8196e4a5c9e17b01bb [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:29:17.845661 24264 raft_consensus.cc:3037] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P ad4cffebbedb4fd8bb14333a335ed442 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:29:17.846310 24264 raft_consensus.cc:2454] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P ad4cffebbedb4fd8bb14333a335ed442 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2612ca8043fd4172a0e1f0e73109e68d in term 1.
I20260925 07:29:17.846421 24130 raft_consensus.cc:2454] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P a023d30cd6fa4a8196e4a5c9e17b01bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2612ca8043fd4172a0e1f0e73109e68d in term 1.
I20260925 07:29:17.846453 23931 leader_election.cc:302] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P 2612ca8043fd4172a0e1f0e73109e68d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2612ca8043fd4172a0e1f0e73109e68d, ad4cffebbedb4fd8bb14333a335ed442; no voters:
I20260925 07:29:17.846544 24332 raft_consensus.cc:2781] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P 2612ca8043fd4172a0e1f0e73109e68d [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:29:17.846665 24332 raft_consensus.cc:686] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P 2612ca8043fd4172a0e1f0e73109e68d [term 1 LEADER]: Becoming Leader. State: Replica: 2612ca8043fd4172a0e1f0e73109e68d, State: Running, Role: LEADER
I20260925 07:29:17.846738 24332 consensus_queue.cc:237] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P 2612ca8043fd4172a0e1f0e73109e68d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad4cffebbedb4fd8bb14333a335ed442" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41669 } } peers { permanent_uuid: "2612ca8043fd4172a0e1f0e73109e68d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42789 } } peers { permanent_uuid: "a023d30cd6fa4a8196e4a5c9e17b01bb" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 38643 } }
I20260925 07:29:17.847463 23851 catalog_manager.cc:5495] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P 2612ca8043fd4172a0e1f0e73109e68d reported cstate change: term changed from 0 to 1, leader changed from <none> to 2612ca8043fd4172a0e1f0e73109e68d (127.5.158.65). New cstate: current_term: 1 leader_uuid: "2612ca8043fd4172a0e1f0e73109e68d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad4cffebbedb4fd8bb14333a335ed442" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2612ca8043fd4172a0e1f0e73109e68d" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 42789 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a023d30cd6fa4a8196e4a5c9e17b01bb" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 38643 } health_report { overall_health: UNKNOWN } } }
I20260925 07:29:17.942236 24339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:17.940227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59236 (local address 127.5.158.65:42789)
0925 07:29:17.940337 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:17.940339 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:17.940346 (+ 7us) server_negotiation.cc:407] Connection header received
0925 07:29:17.940373 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:17.940374 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:17.940388 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:17.940454 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 07:29:17.940576 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:17.940997 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:17.941614 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:17.941770 (+ 156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:17.941815 (+ 45us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:17.941931 (+ 116us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 07:29:17.942006 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 07:29:17.942066 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:17.942109 (+ 43us) server_negotiation.cc:299] Negotiation successful
0925 07:29:17.942137 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":68,"threads_started":1}
I20260925 07:29:17.943769 24264 raft_consensus.cc:1261] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P ad4cffebbedb4fd8bb14333a335ed442 [term 1 FOLLOWER]: Refusing update from remote peer 2612ca8043fd4172a0e1f0e73109e68d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:29:17.943774 24130 raft_consensus.cc:1261] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P a023d30cd6fa4a8196e4a5c9e17b01bb [term 1 FOLLOWER]: Refusing update from remote peer 2612ca8043fd4172a0e1f0e73109e68d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:29:17.944083 24335 consensus_queue.cc:1035] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P 2612ca8043fd4172a0e1f0e73109e68d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad4cffebbedb4fd8bb14333a335ed442" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:29:17.944209 24332 consensus_queue.cc:1035] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P 2612ca8043fd4172a0e1f0e73109e68d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a023d30cd6fa4a8196e4a5c9e17b01bb" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 38643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:29:17.946637 24343 mvcc.cc:205] Tried to move back new op lower bound from 7333156282135191552 to 7333156281740582912. Current Snapshot: MvccSnapshot[applied={T|T < 7333156282135191552}]
I20260925 07:29:17.946817 24344 mvcc.cc:205] Tried to move back new op lower bound from 7333156282135191552 to 7333156281740582912. Current Snapshot: MvccSnapshot[applied={T|T < 7333156282135191552}]
I20260925 07:29:17.948014 23851 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46280:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 07:29:17.948099 23851 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46280:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 07:29:17.948725 23851 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f35e5529d85a4803be239e6a65315030: Sending DeleteTablet for 3 replicas of tablet 1ff2cd2c8f1d401d82ae7a88e606bf27
I20260925 07:29:17.949021 24110 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ff2cd2c8f1d401d82ae7a88e606bf27 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:29:17 UTC) from {username='kudu', principal='kudu/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:56842
I20260925 07:29:17.949057 24244 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ff2cd2c8f1d401d82ae7a88e606bf27 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:29:17 UTC) from {username='kudu', principal='kudu/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:39024
I20260925 07:29:17.949262 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 23913
I20260925 07:29:17.949251 23976 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ff2cd2c8f1d401d82ae7a88e606bf27 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:29:17 UTC) from {username='kudu', principal='kudu/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:51086
I20260925 07:29:17.949338 24352 tablet_replica.cc:330] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P a023d30cd6fa4a8196e4a5c9e17b01bb: stopping tablet replica
I20260925 07:29:17.949417 24352 raft_consensus.cc:2229] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P a023d30cd6fa4a8196e4a5c9e17b01bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 07:29:17.949494 24352 raft_consensus.cc:2258] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P a023d30cd6fa4a8196e4a5c9e17b01bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:29:17.949610 24353 tablet_replica.cc:330] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P ad4cffebbedb4fd8bb14333a335ed442: stopping tablet replica
I20260925 07:29:17.949712 24353 raft_consensus.cc:2229] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P ad4cffebbedb4fd8bb14333a335ed442 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 07:29:17.949793 24353 raft_consensus.cc:2258] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P ad4cffebbedb4fd8bb14333a335ed442 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:29:17.950160 24353 ts_tablet_manager.cc:1905] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P ad4cffebbedb4fd8bb14333a335ed442: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:29:17.951411 24352 ts_tablet_manager.cc:1905] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P a023d30cd6fa4a8196e4a5c9e17b01bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:29:17.951629 24353 ts_tablet_manager.cc:1918] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P ad4cffebbedb4fd8bb14333a335ed442: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 07:29:17.951702 24353 log.cc:1196] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P ad4cffebbedb4fd8bb14333a335ed442: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/ts-2/wal/wals/1ff2cd2c8f1d401d82ae7a88e606bf27
I20260925 07:29:17.951939 24353 ts_tablet_manager.cc:1939] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P ad4cffebbedb4fd8bb14333a335ed442: Deleting consensus metadata
I20260925 07:29:17.952278 23837 catalog_manager.cc:4841] TS ad4cffebbedb4fd8bb14333a335ed442 (127.5.158.67:41669): tablet 1ff2cd2c8f1d401d82ae7a88e606bf27 (table test-table [id=5b2b9babecad4b6a82488b14fe079681]) successfully deleted
I20260925 07:29:17.953151 24352 ts_tablet_manager.cc:1918] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P a023d30cd6fa4a8196e4a5c9e17b01bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 07:29:17.953219 24352 log.cc:1196] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P a023d30cd6fa4a8196e4a5c9e17b01bb: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790321110323796-5753-0/minicluster-data/ts-1/wal/wals/1ff2cd2c8f1d401d82ae7a88e606bf27
I20260925 07:29:17.953438 24352 ts_tablet_manager.cc:1939] T 1ff2cd2c8f1d401d82ae7a88e606bf27 P a023d30cd6fa4a8196e4a5c9e17b01bb: Deleting consensus metadata
I20260925 07:29:17.953783 23836 catalog_manager.cc:4841] TS a023d30cd6fa4a8196e4a5c9e17b01bb (127.5.158.66:38643): tablet 1ff2cd2c8f1d401d82ae7a88e606bf27 (table test-table [id=5b2b9babecad4b6a82488b14fe079681]) successfully deleted
W20260925 07:29:17.955283 23837 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.158.65:42789 (error 108)
I20260925 07:29:17.955483 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 24047
I20260925 07:29:17.956089 24355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:17.955644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:42789 (local address 127.0.0.1:59252)
0925 07:29:17.956007 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:17.956045 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.65:42789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":37,"threads_started":1}
W20260925 07:29:17.956229 23837 catalog_manager.cc:4584] TS 2612ca8043fd4172a0e1f0e73109e68d (127.5.158.65:42789): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1ff2cd2c8f1d401d82ae7a88e606bf27: Network error: Client connection negotiation failed: client connection to 127.5.158.65:42789: connect: Connection refused (error 111)
I20260925 07:29:17.959956 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 24181
I20260925 07:29:17.964530 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 23819
2026-09-25T07:29:17Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2214 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-25T07:29:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T07:29:17Z Disabled control of system clock
I20260925 07:29:17.989550 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:44051
--webserver_interface=127.5.158.126
--webserver_port=0
--builtin_ntp_servers=127.5.158.84:37491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.158.126:44051
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 07:29:18.062377 24359 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:29:18.062592 24359 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:29:18.062623 24359 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:29:18.063650 24359 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:29:18.063706 24359 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:29:18.063725 24359 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:29:18.063742 24359 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:29:18.063768 24359 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:29:18.064888 24359 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:37491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.158.126:44051
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:44051
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24359
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:29:18.065130 24359 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:29:18.065369 24359 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:29:18.069325 24364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:18.069401 24365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:18.069361 24367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:18.069578 24359 server_base.cc:1034] running on GCE node
I20260925 07:29:18.069864 24359 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:29:18.070137 24359 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:29:18.071300 24359 hybrid_clock.cc:648] HybridClock initialized: now 1790321358071287 us; error 41 us; skew 500 ppm
I20260925 07:29:18.072363 24359 webserver.cc:460] Webserver started at http://127.5.158.126:38621/ using document root <none> and password file <none>
I20260925 07:29:18.072597 24359 fs_manager.cc:359] Metadata directory not provided
I20260925 07:29:18.072664 24359 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:29:18.072809 24359 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:29:18.073782 24359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/master-0/data/instance:
uuid: "788c882e23f1442597d942d13b2e8808"
format_stamp: "Formatted at 2026-09-25 07:29:18 on dist-test-slave-1drq"
server_key: "058074109e66d6496c7171ce347da189"
server_key_iv: "722dc849a74d49e4b7d0b5b1a08b78d0"
server_key_version: "encryptionkey@0"
I20260925 07:29:18.074131 24359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/master-0/wal/instance:
uuid: "788c882e23f1442597d942d13b2e8808"
format_stamp: "Formatted at 2026-09-25 07:29:18 on dist-test-slave-1drq"
server_key: "058074109e66d6496c7171ce347da189"
server_key_iv: "722dc849a74d49e4b7d0b5b1a08b78d0"
server_key_version: "encryptionkey@0"
I20260925 07:29:18.075277 24359 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 07:29:18.076599 24373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:18.076793 24359 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 07:29:18.076886 24359 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "788c882e23f1442597d942d13b2e8808"
format_stamp: "Formatted at 2026-09-25 07:29:18 on dist-test-slave-1drq"
server_key: "058074109e66d6496c7171ce347da189"
server_key_iv: "722dc849a74d49e4b7d0b5b1a08b78d0"
server_key_version: "encryptionkey@0"
I20260925 07:29:18.076969 24359 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:29:18.085950 24359 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:29:18.086205 24359 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:29:18.086351 24359 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:29:18.089895 24359 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.126:44051
I20260925 07:29:18.089921 24425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.126:44051 every 8 connection(s)
I20260925 07:29:18.090217 24359 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:29:18.091028 24426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:29:18.092882 24426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 788c882e23f1442597d942d13b2e8808: Bootstrap starting.
I20260925 07:29:18.093284 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 24359
I20260925 07:29:18.093385 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/master-0/wal/instance
I20260925 07:29:18.093484 24426 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 788c882e23f1442597d942d13b2e8808: Neither blocks nor log segments found. Creating new log.
I20260925 07:29:18.093582 5753 external_mini_cluster.cc:1428] Setting key 2faa5e3ab44cfc63465b5be41e578ba3
I20260925 07:29:18.093776 24426 log.cc:824] T 00000000000000000000000000000000 P 788c882e23f1442597d942d13b2e8808: Log is configured to *not* fsync() on all Append() calls
I20260925 07:29:18.094460 24426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 788c882e23f1442597d942d13b2e8808: No bootstrap required, opened a new log
I20260925 07:29:18.096678 24426 raft_consensus.cc:348] T 00000000000000000000000000000000 P 788c882e23f1442597d942d13b2e8808 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "788c882e23f1442597d942d13b2e8808" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 44051 } }
I20260925 07:29:18.096781 24426 raft_consensus.cc:374] T 00000000000000000000000000000000 P 788c882e23f1442597d942d13b2e8808 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:29:18.096799 24426 raft_consensus.cc:729] T 00000000000000000000000000000000 P 788c882e23f1442597d942d13b2e8808 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 788c882e23f1442597d942d13b2e8808, State: Initialized, Role: FOLLOWER
I20260925 07:29:18.096927 24426 consensus_queue.cc:260] T 00000000000000000000000000000000 P 788c882e23f1442597d942d13b2e8808 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "788c882e23f1442597d942d13b2e8808" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 44051 } }
I20260925 07:29:18.096998 24426 raft_consensus.cc:388] T 00000000000000000000000000000000 P 788c882e23f1442597d942d13b2e8808 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 07:29:18.097039 24426 raft_consensus.cc:482] T 00000000000000000000000000000000 P 788c882e23f1442597d942d13b2e8808 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 07:29:18.097103 24426 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 788c882e23f1442597d942d13b2e8808 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:29:18.097677 24426 raft_consensus.cc:504] T 00000000000000000000000000000000 P 788c882e23f1442597d942d13b2e8808 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "788c882e23f1442597d942d13b2e8808" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 44051 } }
I20260925 07:29:18.097787 24426 leader_election.cc:302] T 00000000000000000000000000000000 P 788c882e23f1442597d942d13b2e8808 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 788c882e23f1442597d942d13b2e8808; no voters:
I20260925 07:29:18.097936 24426 leader_election.cc:288] T 00000000000000000000000000000000 P 788c882e23f1442597d942d13b2e8808 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 07:29:18.097965 24431 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 788c882e23f1442597d942d13b2e8808 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:29:18.098112 24431 raft_consensus.cc:686] T 00000000000000000000000000000000 P 788c882e23f1442597d942d13b2e8808 [term 1 LEADER]: Becoming Leader. State: Replica: 788c882e23f1442597d942d13b2e8808, State: Running, Role: LEADER
I20260925 07:29:18.098220 24426 sys_catalog.cc:564] T 00000000000000000000000000000000 P 788c882e23f1442597d942d13b2e8808 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:29:18.098201 24431 consensus_queue.cc:237] T 00000000000000000000000000000000 P 788c882e23f1442597d942d13b2e8808 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "788c882e23f1442597d942d13b2e8808" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 44051 } }
I20260925 07:29:18.098631 24433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 788c882e23f1442597d942d13b2e8808 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 788c882e23f1442597d942d13b2e8808. Latest consensus state: current_term: 1 leader_uuid: "788c882e23f1442597d942d13b2e8808" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "788c882e23f1442597d942d13b2e8808" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 44051 } } }
I20260925 07:29:18.098712 24432 sys_catalog.cc:455] T 00000000000000000000000000000000 P 788c882e23f1442597d942d13b2e8808 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "788c882e23f1442597d942d13b2e8808" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "788c882e23f1442597d942d13b2e8808" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 44051 } } }
I20260925 07:29:18.098775 24433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 788c882e23f1442597d942d13b2e8808 [sys.catalog]: This master's current role is: LEADER
I20260925 07:29:18.099063 24430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:18.094029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49332 (local address 127.5.158.126:44051)
0925 07:29:18.094196 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:18.094203 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:18.094233 (+ 30us) server_negotiation.cc:407] Connection header received
0925 07:29:18.094755 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:18.094762 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:18.094828 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:18.094963 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:18.095103 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:18.096402 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:18.097286 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:18.097633 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:18.097771 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:18.098378 (+ 607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:18.098381 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:18.098382 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:29:18.098550 (+ 168us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:18.098590 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:29:18.098603 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:18.098665 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:18.098752 (+ 87us) server_negotiation.cc:299] Negotiation successful
0925 07:29:18.098820 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":57,"threads_started":1}
I20260925 07:29:18.098862 24432 sys_catalog.cc:458] T 00000000000000000000000000000000 P 788c882e23f1442597d942d13b2e8808 [sys.catalog]: This master's current role is: LEADER
I20260925 07:29:18.099308 24440 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 07:29:18.099825 24440 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 07:29:18.101361 24440 catalog_manager.cc:1347] Generated new cluster ID: a928f36a963c41daa4e91698e3f6066a
I20260925 07:29:18.101420 24440 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 07:29:18.123071 24440 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 07:29:18.123554 24440 catalog_manager.cc:1504] Loading token signing keys...
I20260925 07:29:18.126627 24440 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 788c882e23f1442597d942d13b2e8808: Generated new TSK 0
I20260925 07:29:18.126767 24440 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 07:29:18.132158 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.65:0
--local_ip_for_outbound_sockets=127.5.158.65
--webserver_interface=127.5.158.65
--webserver_port=0
--tserver_master_addrs=127.5.158.126:44051
--builtin_ntp_servers=127.5.158.84:37491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 07:29:18.206234 24450 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:29:18.206390 24450 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:29:18.206408 24450 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:29:18.207383 24450 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:29:18.207415 24450 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:29:18.207437 24450 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.65
I20260925 07:29:18.208606 24450 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:37491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.158.65
--webserver_port=0
--tserver_master_addrs=127.5.158.126:44051
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24450
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.65
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:29:18.208804 24450 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:29:18.209056 24450 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:29:18.213752 24458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:18.213783 24456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:18.213765 24455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:18.214008 24450 server_base.cc:1034] running on GCE node
I20260925 07:29:18.214165 24450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:29:18.214389 24450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:29:18.215528 24450 hybrid_clock.cc:648] HybridClock initialized: now 1790321358215499 us; error 42 us; skew 500 ppm
I20260925 07:29:18.216552 24450 webserver.cc:460] Webserver started at http://127.5.158.65:44037/ using document root <none> and password file <none>
I20260925 07:29:18.216758 24450 fs_manager.cc:359] Metadata directory not provided
I20260925 07:29:18.216797 24450 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:29:18.216913 24450 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:29:18.217904 24450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/ts-0/data/instance:
uuid: "7c8b76f87ab14a4e94f32e9f8c6d3221"
format_stamp: "Formatted at 2026-09-25 07:29:18 on dist-test-slave-1drq"
server_key: "53c3a4a4413873a9a7bb7c29c0381f5d"
server_key_iv: "184acd0d5fa9ec36c0263e364d23ef5e"
server_key_version: "encryptionkey@0"
I20260925 07:29:18.218235 24450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/ts-0/wal/instance:
uuid: "7c8b76f87ab14a4e94f32e9f8c6d3221"
format_stamp: "Formatted at 2026-09-25 07:29:18 on dist-test-slave-1drq"
server_key: "53c3a4a4413873a9a7bb7c29c0381f5d"
server_key_iv: "184acd0d5fa9ec36c0263e364d23ef5e"
server_key_version: "encryptionkey@0"
I20260925 07:29:18.219482 24450 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 07:29:18.220166 24464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:18.220358 24450 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:29:18.220423 24450 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/ts-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/ts-0/wal
uuid: "7c8b76f87ab14a4e94f32e9f8c6d3221"
format_stamp: "Formatted at 2026-09-25 07:29:18 on dist-test-slave-1drq"
server_key: "53c3a4a4413873a9a7bb7c29c0381f5d"
server_key_iv: "184acd0d5fa9ec36c0263e364d23ef5e"
server_key_version: "encryptionkey@0"
I20260925 07:29:18.220510 24450 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:29:18.232924 24450 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:29:18.233223 24450 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:29:18.233314 24450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:29:18.233521 24450 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:29:18.233799 24450 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:29:18.233829 24450 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:18.233875 24450 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:29:18.233899 24450 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:18.238967 24450 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.65:43359
I20260925 07:29:18.239037 24577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.65:43359 every 8 connection(s)
I20260925 07:29:18.239336 24450 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
I20260925 07:29:18.243832 24430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:18.240011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:49705 (local address 127.5.158.126:44051)
0925 07:29:18.240057 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:18.240060 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:18.240299 (+ 239us) server_negotiation.cc:407] Connection header received
0925 07:29:18.240697 (+ 398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:18.240698 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:18.240707 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:18.240755 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:18.241439 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:18.241913 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:18.242789 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:18.242960 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:18.243006 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:18.243653 (+ 647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:18.243654 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:18.243655 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:29:18.243658 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:18.243669 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:29:18.243672 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:18.243708 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:18.243763 (+ 55us) server_negotiation.cc:299] Negotiation successful
0925 07:29:18.243780 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 07:29:18.244043 24580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:18.240015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:44051 (local address 127.5.158.65:49705)
0925 07:29:18.240172 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:18.240204 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:18.240623 (+ 419us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:18.240799 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:18.240801 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:18.240808 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:29:18.241361 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:18.241365 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:18.241985 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:18.241986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:18.242717 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:18.242718 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:18.243027 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:18.243028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:18.243132 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:18.243538 (+ 406us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 07:29:18.243538 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:29:18.243561 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 07:29:18.243581 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:29:18.243715 (+ 134us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:29:18.243717 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:29:18.243719 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 07:29:18.243758 (+ 39us) client_negotiation.cc:240] Negotiation successful
0925 07:29:18.243822 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":83,"threads_started":1}
I20260925 07:29:18.244521 24578 heartbeater.cc:347] Connected to a master server at 127.5.158.126:44051
I20260925 07:29:18.244592 24578 heartbeater.cc:464] Registering TS with master...
I20260925 07:29:18.244771 24578 heartbeater.cc:511] Master 127.5.158.126:44051 requested a full tablet report, sending...
I20260925 07:29:18.245254 24390 ts_manager.cc:194] Registered new tserver with Master: 7c8b76f87ab14a4e94f32e9f8c6d3221 (127.5.158.65:43359)
I20260925 07:29:18.245908 24390 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.158.65:49705
I20260925 07:29:18.246320 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 24450
I20260925 07:29:18.246479 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/ts-0/wal/instance
I20260925 07:29:18.246681 5753 external_mini_cluster.cc:1428] Setting key 79e98e8e6b1259838d915603ea123577
I20260925 07:29:18.247463 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.66:0
--local_ip_for_outbound_sockets=127.5.158.66
--webserver_interface=127.5.158.66
--webserver_port=0
--tserver_master_addrs=127.5.158.126:44051
--builtin_ntp_servers=127.5.158.84:37491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 07:29:18.320922 24581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:29:18.321146 24581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:29:18.321206 24581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:29:18.322209 24581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:29:18.322243 24581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:29:18.322265 24581 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.66
I20260925 07:29:18.323520 24581 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:37491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.158.66
--webserver_port=0
--tserver_master_addrs=127.5.158.126:44051
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24581
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.66
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:29:18.323779 24581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:29:18.324052 24581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:29:18.328523 24589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:18.328548 24587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:18.328629 24581 server_base.cc:1034] running on GCE node
W20260925 07:29:18.328531 24586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:18.328941 24581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:29:18.329162 24581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:29:18.330310 24581 hybrid_clock.cc:648] HybridClock initialized: now 1790321358330284 us; error 43 us; skew 500 ppm
I20260925 07:29:18.331444 24581 webserver.cc:460] Webserver started at http://127.5.158.66:34233/ using document root <none> and password file <none>
I20260925 07:29:18.331611 24581 fs_manager.cc:359] Metadata directory not provided
I20260925 07:29:18.331648 24581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:29:18.331735 24581 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:29:18.332612 24581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/ts-1/data/instance:
uuid: "513ad7723e574f918ed20d18aac3e910"
format_stamp: "Formatted at 2026-09-25 07:29:18 on dist-test-slave-1drq"
server_key: "8220074c35ec1465f3ef9eb6b0cd2e10"
server_key_iv: "f5b4d3096c04eeb700b52fb9eb2582f8"
server_key_version: "encryptionkey@0"
I20260925 07:29:18.332898 24581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/ts-1/wal/instance:
uuid: "513ad7723e574f918ed20d18aac3e910"
format_stamp: "Formatted at 2026-09-25 07:29:18 on dist-test-slave-1drq"
server_key: "8220074c35ec1465f3ef9eb6b0cd2e10"
server_key_iv: "f5b4d3096c04eeb700b52fb9eb2582f8"
server_key_version: "encryptionkey@0"
I20260925 07:29:18.334110 24581 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 07:29:18.334698 24595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:18.334837 24581 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:29:18.334894 24581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/ts-1/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/ts-1/wal
uuid: "513ad7723e574f918ed20d18aac3e910"
format_stamp: "Formatted at 2026-09-25 07:29:18 on dist-test-slave-1drq"
server_key: "8220074c35ec1465f3ef9eb6b0cd2e10"
server_key_iv: "f5b4d3096c04eeb700b52fb9eb2582f8"
server_key_version: "encryptionkey@0"
I20260925 07:29:18.334936 24581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:29:18.370573 24581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:29:18.370781 24581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:29:18.370877 24581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:29:18.371040 24581 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:29:18.371265 24581 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:29:18.371295 24581 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:18.371317 24581 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:29:18.371336 24581 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:18.377353 24581 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.66:34677
I20260925 07:29:18.377382 24708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.66:34677 every 8 connection(s)
I20260925 07:29:18.377674 24581 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
I20260925 07:29:18.381920 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 24581
I20260925 07:29:18.382014 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/ts-1/wal/instance
I20260925 07:29:18.382503 5753 external_mini_cluster.cc:1428] Setting key a80a2d661fc63e4fd9c5b49c9ae7043a
I20260925 07:29:18.382778 24430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:18.378376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:33521 (local address 127.5.158.126:44051)
0925 07:29:18.378415 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:18.378418 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:18.378596 (+ 178us) server_negotiation.cc:407] Connection header received
0925 07:29:18.379094 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:18.379096 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:18.379110 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:18.379159 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:18.379957 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:18.380461 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:18.381500 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:18.381684 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:18.381755 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:18.382523 (+ 768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:18.382524 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:18.382524 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:29:18.382530 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:18.382545 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:29:18.382548 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:18.382593 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:18.382681 (+ 88us) server_negotiation.cc:299] Negotiation successful
0925 07:29:18.382704 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 07:29:18.382970 24711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:18.378340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:44051 (local address 127.5.158.66:33521)
0925 07:29:18.378491 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:18.378522 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:18.378987 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:18.379220 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:18.379224 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:18.379233 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:29:18.379863 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:18.379868 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:18.380572 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:18.380573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:18.381418 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:18.381420 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:18.381778 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:18.381780 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:18.381888 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:18.382367 (+ 479us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 07:29:18.382368 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:29:18.382391 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 07:29:18.382411 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:29:18.382593 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:29:18.382598 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:29:18.382601 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 07:29:18.382682 (+ 81us) client_negotiation.cc:240] Negotiation successful
0925 07:29:18.382774 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":62,"threads_started":1}
I20260925 07:29:18.383389 24709 heartbeater.cc:347] Connected to a master server at 127.5.158.126:44051
I20260925 07:29:18.383473 24709 heartbeater.cc:464] Registering TS with master...
I20260925 07:29:18.383565 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.67:0
--local_ip_for_outbound_sockets=127.5.158.67
--webserver_interface=127.5.158.67
--webserver_port=0
--tserver_master_addrs=127.5.158.126:44051
--builtin_ntp_servers=127.5.158.84:37491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 07:29:18.383656 24709 heartbeater.cc:511] Master 127.5.158.126:44051 requested a full tablet report, sending...
I20260925 07:29:18.384063 24390 ts_manager.cc:194] Registered new tserver with Master: 513ad7723e574f918ed20d18aac3e910 (127.5.158.66:34677)
I20260925 07:29:18.384509 24390 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.158.66:33521
W20260925 07:29:18.458356 24712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:29:18.458544 24712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:29:18.458573 24712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:29:18.459628 24712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:29:18.459688 24712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:29:18.459729 24712 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.67
I20260925 07:29:18.461054 24712 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:37491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.158.67
--webserver_port=0
--tserver_master_addrs=127.5.158.126:44051
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24712
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.67
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:29:18.461305 24712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:29:18.461575 24712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:29:18.465728 24720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:18.465725 24717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:18.465725 24712 server_base.cc:1034] running on GCE node
W20260925 07:29:18.465843 24718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:18.466107 24712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:29:18.466274 24712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:29:18.467399 24712 hybrid_clock.cc:648] HybridClock initialized: now 1790321358467388 us; error 23 us; skew 500 ppm
I20260925 07:29:18.468451 24712 webserver.cc:460] Webserver started at http://127.5.158.67:33103/ using document root <none> and password file <none>
I20260925 07:29:18.468658 24712 fs_manager.cc:359] Metadata directory not provided
I20260925 07:29:18.468699 24712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:29:18.468823 24712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:29:18.469851 24712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/ts-2/data/instance:
uuid: "7eff823b07434c13ad5563cd3601dfa0"
format_stamp: "Formatted at 2026-09-25 07:29:18 on dist-test-slave-1drq"
server_key: "c7a03ec812f122205786b2bab9f1b537"
server_key_iv: "056dda44e231b8fd5d5b81f4c3ec7254"
server_key_version: "encryptionkey@0"
I20260925 07:29:18.470180 24712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/ts-2/wal/instance:
uuid: "7eff823b07434c13ad5563cd3601dfa0"
format_stamp: "Formatted at 2026-09-25 07:29:18 on dist-test-slave-1drq"
server_key: "c7a03ec812f122205786b2bab9f1b537"
server_key_iv: "056dda44e231b8fd5d5b81f4c3ec7254"
server_key_version: "encryptionkey@0"
I20260925 07:29:18.471278 24712 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 07:29:18.471860 24726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:18.472018 24712 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 07:29:18.472097 24712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/ts-2/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/ts-2/wal
uuid: "7eff823b07434c13ad5563cd3601dfa0"
format_stamp: "Formatted at 2026-09-25 07:29:18 on dist-test-slave-1drq"
server_key: "c7a03ec812f122205786b2bab9f1b537"
server_key_iv: "056dda44e231b8fd5d5b81f4c3ec7254"
server_key_version: "encryptionkey@0"
I20260925 07:29:18.472182 24712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:29:18.490095 24712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:29:18.490348 24712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:29:18.490489 24712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:29:18.490693 24712 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:29:18.490972 24712 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:29:18.491020 24712 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:18.491048 24712 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:29:18.491073 24712 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:18.496340 24712 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.67:41531
I20260925 07:29:18.496387 24839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.67:41531 every 8 connection(s)
I20260925 07:29:18.496654 24712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
I20260925 07:29:18.497882 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 24712
I20260925 07:29:18.498011 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/ts-2/wal/instance
I20260925 07:29:18.498230 5753 external_mini_cluster.cc:1428] Setting key ed8a14e238db080a7dac989093db9f1d
I20260925 07:29:18.501784 24430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:18.497329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:58159 (local address 127.5.158.126:44051)
0925 07:29:18.497371 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:18.497373 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:18.497585 (+ 212us) server_negotiation.cc:407] Connection header received
0925 07:29:18.498162 (+ 577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:18.498163 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:18.498175 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:18.498254 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:18.499086 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:18.499712 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:18.500661 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:18.500841 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:18.500902 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:18.501593 (+ 691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:18.501595 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:18.501595 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:29:18.501598 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:18.501610 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:29:18.501614 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:18.501653 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:18.501715 (+ 62us) server_negotiation.cc:299] Negotiation successful
0925 07:29:18.501734 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 07:29:18.501960 24842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:18.497283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:44051 (local address 127.5.158.67:58159)
0925 07:29:18.497452 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:18.497485 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:18.498052 (+ 567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:18.498317 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:18.498321 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:18.498331 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:29:18.498984 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:18.498991 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:18.499795 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:18.499796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:18.500548 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:18.500550 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:18.500918 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:18.500919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:18.501016 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:18.501495 (+ 479us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 07:29:18.501496 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:29:18.501519 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 07:29:18.501536 (+ 17us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:29:18.501661 (+ 125us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:29:18.501665 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:29:18.501667 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 07:29:18.501717 (+ 50us) client_negotiation.cc:240] Negotiation successful
0925 07:29:18.501781 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":89,"threads_started":1}
I20260925 07:29:18.502295 24840 heartbeater.cc:347] Connected to a master server at 127.5.158.126:44051
I20260925 07:29:18.502384 24840 heartbeater.cc:464] Registering TS with master...
I20260925 07:29:18.502545 24840 heartbeater.cc:511] Master 127.5.158.126:44051 requested a full tablet report, sending...
I20260925 07:29:18.502897 24390 ts_manager.cc:194] Registered new tserver with Master: 7eff823b07434c13ad5563cd3601dfa0 (127.5.158.67:41531)
I20260925 07:29:18.503229 24390 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.158.67:58159
I20260925 07:29:18.509858 5753 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 07:29:18.513525 24430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:18.511159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49340 (local address 127.5.158.126:44051)
0925 07:29:18.511205 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:18.511207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:18.511289 (+ 82us) server_negotiation.cc:407] Connection header received
0925 07:29:18.511321 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:18.511321 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:18.511329 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:18.511378 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:18.511539 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:18.511933 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:18.512559 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:18.512711 (+ 152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:18.512755 (+ 44us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:18.513344 (+ 589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:18.513344 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:18.513345 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:29:18.513348 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:18.513358 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:29:18.513361 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:18.513425 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:18.513471 (+ 46us) server_negotiation.cc:299] Negotiation successful
0925 07:29:18.513478 (+ 7us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260925 07:29:18.515017 24390 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49340:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 07:29:18.516379 24390 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 07:29:18.521763 24849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:18.518129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.67:41531 (local address 127.0.0.1:51798)
0925 07:29:18.518251 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:18.518284 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:18.518353 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:18.518525 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:18.518526 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:18.518528 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:29:18.518578 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:18.518579 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:18.519636 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:18.519637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:18.520557 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:18.520559 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:18.521516 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:18.521517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:18.521581 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:18.521590 (+ 9us) client_negotiation.cc:769] Sending connection context
0925 07:29:18.521648 (+ 58us) client_negotiation.cc:240] Negotiation successful
0925 07:29:18.521698 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":77,"threads_started":1}
I20260925 07:29:18.521770 24850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:18.518184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51798 (local address 127.5.158.67:41531)
0925 07:29:18.518323 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:18.518339 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:18.518348 (+ 9us) server_negotiation.cc:407] Connection header received
0925 07:29:18.518444 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:18.518446 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:18.518457 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:18.518524 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:29:18.518645 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:18.519557 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:18.520637 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:18.521354 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:18.521423 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:18.521488 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:18.521674 (+ 186us) server_negotiation.cc:299] Negotiation successful
0925 07:29:18.521707 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":68,"threads_started":1}
I20260925 07:29:18.522248 24851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:18.518133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.66:34677 (local address 127.0.0.1:34872)
0925 07:29:18.518453 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:18.518463 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:18.518511 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:18.518914 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:18.518915 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:18.518917 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:29:18.518955 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:18.518956 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:18.520319 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:18.520320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:18.521302 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:18.521304 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:18.522045 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:18.522045 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:18.522098 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:18.522101 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 07:29:18.522151 (+ 50us) client_negotiation.cc:240] Negotiation successful
0925 07:29:18.522188 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":44,"threads_started":1}
I20260925 07:29:18.522266 24852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:18.518472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34872 (local address 127.5.158.66:34677)
0925 07:29:18.518716 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:18.518720 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:18.518727 (+ 7us) server_negotiation.cc:407] Connection header received
0925 07:29:18.518756 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:18.518757 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:18.518767 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:18.518857 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:29:18.519020 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:18.520220 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:18.521377 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:18.521949 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:18.522040 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:18.522117 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:18.522164 (+ 47us) server_negotiation.cc:299] Negotiation successful
0925 07:29:18.522197 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":73,"threads_started":1}
I20260925 07:29:18.522696 24848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:18.518302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:43359 (local address 127.0.0.1:40512)
0925 07:29:18.519057 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:18.519065 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:18.519104 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:18.519331 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:18.519332 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:18.519334 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:29:18.519366 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:18.519367 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:18.520542 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:18.520543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:18.521674 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:18.521676 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:18.522510 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:18.522511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:18.522575 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:18.522578 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 07:29:18.522624 (+ 46us) client_negotiation.cc:240] Negotiation successful
0925 07:29:18.522659 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":742,"thread_start_us":85,"threads_started":1}
I20260925 07:29:18.522590 24774 tablet_service.cc:1467] Processing CreateTablet for tablet c6bec901fcec4fb9b6f5477439d16333 (DEFAULT_TABLE table=test-table [id=2762e313217f4191b6e54765980e5687]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:29:18.522735 24853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:18.518698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40512 (local address 127.5.158.65:43359)
0925 07:29:18.518866 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:18.518869 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:18.519154 (+ 285us) server_negotiation.cc:407] Connection header received
0925 07:29:18.519195 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:18.519197 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:18.519211 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:18.519273 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:29:18.519452 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:18.520462 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:18.521745 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:18.522430 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:18.522497 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:18.522568 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:18.522635 (+ 67us) server_negotiation.cc:299] Negotiation successful
0925 07:29:18.522668 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":86,"threads_started":1}
I20260925 07:29:18.522946 24774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6bec901fcec4fb9b6f5477439d16333. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:29:18.525280 24854 tablet_bootstrap.cc:492] T c6bec901fcec4fb9b6f5477439d16333 P 7eff823b07434c13ad5563cd3601dfa0: Bootstrap starting.
I20260925 07:29:18.525643 24643 tablet_service.cc:1467] Processing CreateTablet for tablet c6bec901fcec4fb9b6f5477439d16333 (DEFAULT_TABLE table=test-table [id=2762e313217f4191b6e54765980e5687]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:29:18.525854 24854 tablet_bootstrap.cc:654] T c6bec901fcec4fb9b6f5477439d16333 P 7eff823b07434c13ad5563cd3601dfa0: Neither blocks nor log segments found. Creating new log.
I20260925 07:29:18.525930 24643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6bec901fcec4fb9b6f5477439d16333. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:29:18.526160 24854 log.cc:824] T c6bec901fcec4fb9b6f5477439d16333 P 7eff823b07434c13ad5563cd3601dfa0: Log is configured to *not* fsync() on all Append() calls
I20260925 07:29:18.526760 24854 tablet_bootstrap.cc:492] T c6bec901fcec4fb9b6f5477439d16333 P 7eff823b07434c13ad5563cd3601dfa0: No bootstrap required, opened a new log
I20260925 07:29:18.526837 24854 ts_tablet_manager.cc:1397] T c6bec901fcec4fb9b6f5477439d16333 P 7eff823b07434c13ad5563cd3601dfa0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 07:29:18.528156 24856 tablet_bootstrap.cc:492] T c6bec901fcec4fb9b6f5477439d16333 P 513ad7723e574f918ed20d18aac3e910: Bootstrap starting.
I20260925 07:29:18.528669 24854 raft_consensus.cc:348] T c6bec901fcec4fb9b6f5477439d16333 P 7eff823b07434c13ad5563cd3601dfa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eff823b07434c13ad5563cd3601dfa0" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41531 } } peers { permanent_uuid: "7c8b76f87ab14a4e94f32e9f8c6d3221" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 43359 } } peers { permanent_uuid: "513ad7723e574f918ed20d18aac3e910" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 34677 } }
I20260925 07:29:18.528795 24856 tablet_bootstrap.cc:654] T c6bec901fcec4fb9b6f5477439d16333 P 513ad7723e574f918ed20d18aac3e910: Neither blocks nor log segments found. Creating new log.
I20260925 07:29:18.528825 24854 raft_consensus.cc:374] T c6bec901fcec4fb9b6f5477439d16333 P 7eff823b07434c13ad5563cd3601dfa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:29:18.528859 24854 raft_consensus.cc:729] T c6bec901fcec4fb9b6f5477439d16333 P 7eff823b07434c13ad5563cd3601dfa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eff823b07434c13ad5563cd3601dfa0, State: Initialized, Role: FOLLOWER
I20260925 07:29:18.528788 24512 tablet_service.cc:1467] Processing CreateTablet for tablet c6bec901fcec4fb9b6f5477439d16333 (DEFAULT_TABLE table=test-table [id=2762e313217f4191b6e54765980e5687]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:29:18.529016 24854 consensus_queue.cc:260] T c6bec901fcec4fb9b6f5477439d16333 P 7eff823b07434c13ad5563cd3601dfa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eff823b07434c13ad5563cd3601dfa0" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41531 } } peers { permanent_uuid: "7c8b76f87ab14a4e94f32e9f8c6d3221" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 43359 } } peers { permanent_uuid: "513ad7723e574f918ed20d18aac3e910" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 34677 } }
I20260925 07:29:18.529075 24856 log.cc:824] T c6bec901fcec4fb9b6f5477439d16333 P 513ad7723e574f918ed20d18aac3e910: Log is configured to *not* fsync() on all Append() calls
I20260925 07:29:18.529161 24512 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6bec901fcec4fb9b6f5477439d16333. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:29:18.529300 24854 ts_tablet_manager.cc:1428] T c6bec901fcec4fb9b6f5477439d16333 P 7eff823b07434c13ad5563cd3601dfa0: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 07:29:18.529455 24840 heartbeater.cc:503] Master 127.5.158.126:44051 was elected leader, sending a full tablet report...
I20260925 07:29:18.529939 24856 tablet_bootstrap.cc:492] T c6bec901fcec4fb9b6f5477439d16333 P 513ad7723e574f918ed20d18aac3e910: No bootstrap required, opened a new log
I20260925 07:29:18.529990 24856 ts_tablet_manager.cc:1397] T c6bec901fcec4fb9b6f5477439d16333 P 513ad7723e574f918ed20d18aac3e910: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260925 07:29:18.532044 24859 tablet_bootstrap.cc:492] T c6bec901fcec4fb9b6f5477439d16333 P 7c8b76f87ab14a4e94f32e9f8c6d3221: Bootstrap starting.
I20260925 07:29:18.531941 24856 raft_consensus.cc:348] T c6bec901fcec4fb9b6f5477439d16333 P 513ad7723e574f918ed20d18aac3e910 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eff823b07434c13ad5563cd3601dfa0" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41531 } } peers { permanent_uuid: "7c8b76f87ab14a4e94f32e9f8c6d3221" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 43359 } } peers { permanent_uuid: "513ad7723e574f918ed20d18aac3e910" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 34677 } }
I20260925 07:29:18.532100 24856 raft_consensus.cc:374] T c6bec901fcec4fb9b6f5477439d16333 P 513ad7723e574f918ed20d18aac3e910 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:29:18.532143 24856 raft_consensus.cc:729] T c6bec901fcec4fb9b6f5477439d16333 P 513ad7723e574f918ed20d18aac3e910 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 513ad7723e574f918ed20d18aac3e910, State: Initialized, Role: FOLLOWER
I20260925 07:29:18.532258 24856 consensus_queue.cc:260] T c6bec901fcec4fb9b6f5477439d16333 P 513ad7723e574f918ed20d18aac3e910 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eff823b07434c13ad5563cd3601dfa0" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41531 } } peers { permanent_uuid: "7c8b76f87ab14a4e94f32e9f8c6d3221" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 43359 } } peers { permanent_uuid: "513ad7723e574f918ed20d18aac3e910" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 34677 } }
I20260925 07:29:18.532516 24856 ts_tablet_manager.cc:1428] T c6bec901fcec4fb9b6f5477439d16333 P 513ad7723e574f918ed20d18aac3e910: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260925 07:29:18.532568 24709 heartbeater.cc:503] Master 127.5.158.126:44051 was elected leader, sending a full tablet report...
I20260925 07:29:18.532660 24859 tablet_bootstrap.cc:654] T c6bec901fcec4fb9b6f5477439d16333 P 7c8b76f87ab14a4e94f32e9f8c6d3221: Neither blocks nor log segments found. Creating new log.
I20260925 07:29:18.532950 24859 log.cc:824] T c6bec901fcec4fb9b6f5477439d16333 P 7c8b76f87ab14a4e94f32e9f8c6d3221: Log is configured to *not* fsync() on all Append() calls
I20260925 07:29:18.533630 24859 tablet_bootstrap.cc:492] T c6bec901fcec4fb9b6f5477439d16333 P 7c8b76f87ab14a4e94f32e9f8c6d3221: No bootstrap required, opened a new log
I20260925 07:29:18.533675 24859 ts_tablet_manager.cc:1397] T c6bec901fcec4fb9b6f5477439d16333 P 7c8b76f87ab14a4e94f32e9f8c6d3221: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 07:29:18.535274 24859 raft_consensus.cc:348] T c6bec901fcec4fb9b6f5477439d16333 P 7c8b76f87ab14a4e94f32e9f8c6d3221 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eff823b07434c13ad5563cd3601dfa0" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41531 } } peers { permanent_uuid: "7c8b76f87ab14a4e94f32e9f8c6d3221" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 43359 } } peers { permanent_uuid: "513ad7723e574f918ed20d18aac3e910" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 34677 } }
I20260925 07:29:18.535418 24859 raft_consensus.cc:374] T c6bec901fcec4fb9b6f5477439d16333 P 7c8b76f87ab14a4e94f32e9f8c6d3221 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:29:18.535458 24859 raft_consensus.cc:729] T c6bec901fcec4fb9b6f5477439d16333 P 7c8b76f87ab14a4e94f32e9f8c6d3221 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c8b76f87ab14a4e94f32e9f8c6d3221, State: Initialized, Role: FOLLOWER
I20260925 07:29:18.535574 24859 consensus_queue.cc:260] T c6bec901fcec4fb9b6f5477439d16333 P 7c8b76f87ab14a4e94f32e9f8c6d3221 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eff823b07434c13ad5563cd3601dfa0" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41531 } } peers { permanent_uuid: "7c8b76f87ab14a4e94f32e9f8c6d3221" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 43359 } } peers { permanent_uuid: "513ad7723e574f918ed20d18aac3e910" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 34677 } }
I20260925 07:29:18.535813 24859 ts_tablet_manager.cc:1428] T c6bec901fcec4fb9b6f5477439d16333 P 7c8b76f87ab14a4e94f32e9f8c6d3221: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 07:29:18.535843 24578 heartbeater.cc:503] Master 127.5.158.126:44051 was elected leader, sending a full tablet report...
W20260925 07:29:18.628163 24710 tablet.cc:2369] T c6bec901fcec4fb9b6f5477439d16333 P 513ad7723e574f918ed20d18aac3e910: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 07:29:18.739998 24579 tablet.cc:2369] T c6bec901fcec4fb9b6f5477439d16333 P 7c8b76f87ab14a4e94f32e9f8c6d3221: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 07:29:18.747119 24841 tablet.cc:2369] T c6bec901fcec4fb9b6f5477439d16333 P 7eff823b07434c13ad5563cd3601dfa0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 07:29:18.825233 24860 raft_consensus.cc:482] T c6bec901fcec4fb9b6f5477439d16333 P 513ad7723e574f918ed20d18aac3e910 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 07:29:18.825325 24860 raft_consensus.cc:504] T c6bec901fcec4fb9b6f5477439d16333 P 513ad7723e574f918ed20d18aac3e910 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eff823b07434c13ad5563cd3601dfa0" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41531 } } peers { permanent_uuid: "7c8b76f87ab14a4e94f32e9f8c6d3221" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 43359 } } peers { permanent_uuid: "513ad7723e574f918ed20d18aac3e910" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 34677 } }
I20260925 07:29:18.825672 24860 leader_election.cc:288] T c6bec901fcec4fb9b6f5477439d16333 P 513ad7723e574f918ed20d18aac3e910 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7eff823b07434c13ad5563cd3601dfa0 (127.5.158.67:41531), 7c8b76f87ab14a4e94f32e9f8c6d3221 (127.5.158.65:43359)
I20260925 07:29:18.828934 24850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:18.825969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:55275 (local address 127.5.158.67:41531)
0925 07:29:18.826025 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:18.826028 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:18.826037 (+ 9us) server_negotiation.cc:407] Connection header received
0925 07:29:18.826065 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:18.826065 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:18.826074 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:18.826127 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:29:18.826292 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:18.826930 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:18.828088 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:18.828587 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:18.828656 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:18.828663 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:18.828837 (+ 174us) server_negotiation.cc:299] Negotiation successful
0925 07:29:18.828872 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 07:29:18.828934 24853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:18.826048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:55397 (local address 127.5.158.65:43359)
0925 07:29:18.826085 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:18.826087 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:18.826210 (+ 123us) server_negotiation.cc:407] Connection header received
0925 07:29:18.826280 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:18.826281 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:18.826293 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:18.826339 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:29:18.826463 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:18.827043 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:18.828087 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:18.828591 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:18.828656 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:18.828663 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:18.828837 (+ 174us) server_negotiation.cc:299] Negotiation successful
0925 07:29:18.828872 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 07:29:18.828946 24711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:18.825868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.67:41531 (local address 127.5.158.66:55275)
0925 07:29:18.825920 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:18.825931 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:18.825983 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:18.826149 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:18.826150 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:18.826152 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:29:18.826227 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:18.826229 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:18.826992 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:18.826992 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:18.828004 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:18.828007 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:18.828666 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:18.828667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:18.828762 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:18.828768 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 07:29:18.828819 (+ 51us) client_negotiation.cc:240] Negotiation successful
0925 07:29:18.828864 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":32}
I20260925 07:29:18.828953 24863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:18.825966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:43359 (local address 127.5.158.66:55397)
0925 07:29:18.826129 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:18.826138 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:18.826187 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:18.826356 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:18.826356 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:18.826358 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:29:18.826402 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:18.826403 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:18.827100 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:18.827101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:18.828006 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:18.828007 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:18.828669 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:18.828670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:18.828762 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:18.828768 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 07:29:18.828819 (+ 51us) client_negotiation.cc:240] Negotiation successful
0925 07:29:18.828863 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":8,"thread_start_us":99,"threads_started":1}
I20260925 07:29:18.829324 24794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6bec901fcec4fb9b6f5477439d16333" candidate_uuid: "513ad7723e574f918ed20d18aac3e910" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eff823b07434c13ad5563cd3601dfa0" is_pre_election: true
I20260925 07:29:18.829540 24794 raft_consensus.cc:2454] T c6bec901fcec4fb9b6f5477439d16333 P 7eff823b07434c13ad5563cd3601dfa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 513ad7723e574f918ed20d18aac3e910 in term 0.
I20260925 07:29:18.829643 24532 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6bec901fcec4fb9b6f5477439d16333" candidate_uuid: "513ad7723e574f918ed20d18aac3e910" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c8b76f87ab14a4e94f32e9f8c6d3221" is_pre_election: true
I20260925 07:29:18.829754 24598 leader_election.cc:302] T c6bec901fcec4fb9b6f5477439d16333 P 513ad7723e574f918ed20d18aac3e910 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 513ad7723e574f918ed20d18aac3e910, 7eff823b07434c13ad5563cd3601dfa0; no voters:
I20260925 07:29:18.829797 24532 raft_consensus.cc:2454] T c6bec901fcec4fb9b6f5477439d16333 P 7c8b76f87ab14a4e94f32e9f8c6d3221 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 513ad7723e574f918ed20d18aac3e910 in term 0.
I20260925 07:29:18.829860 24860 raft_consensus.cc:2781] T c6bec901fcec4fb9b6f5477439d16333 P 513ad7723e574f918ed20d18aac3e910 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 07:29:18.829936 24860 raft_consensus.cc:482] T c6bec901fcec4fb9b6f5477439d16333 P 513ad7723e574f918ed20d18aac3e910 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 07:29:18.829963 24860 raft_consensus.cc:3037] T c6bec901fcec4fb9b6f5477439d16333 P 513ad7723e574f918ed20d18aac3e910 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:29:18.830653 24860 raft_consensus.cc:504] T c6bec901fcec4fb9b6f5477439d16333 P 513ad7723e574f918ed20d18aac3e910 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eff823b07434c13ad5563cd3601dfa0" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41531 } } peers { permanent_uuid: "7c8b76f87ab14a4e94f32e9f8c6d3221" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 43359 } } peers { permanent_uuid: "513ad7723e574f918ed20d18aac3e910" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 34677 } }
I20260925 07:29:18.830762 24860 leader_election.cc:288] T c6bec901fcec4fb9b6f5477439d16333 P 513ad7723e574f918ed20d18aac3e910 [CANDIDATE]: Term 1 election: Requested vote from peers 7eff823b07434c13ad5563cd3601dfa0 (127.5.158.67:41531), 7c8b76f87ab14a4e94f32e9f8c6d3221 (127.5.158.65:43359)
I20260925 07:29:18.830914 24794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6bec901fcec4fb9b6f5477439d16333" candidate_uuid: "513ad7723e574f918ed20d18aac3e910" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eff823b07434c13ad5563cd3601dfa0"
I20260925 07:29:18.830919 24532 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6bec901fcec4fb9b6f5477439d16333" candidate_uuid: "513ad7723e574f918ed20d18aac3e910" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c8b76f87ab14a4e94f32e9f8c6d3221"
I20260925 07:29:18.831000 24532 raft_consensus.cc:3037] T c6bec901fcec4fb9b6f5477439d16333 P 7c8b76f87ab14a4e94f32e9f8c6d3221 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:29:18.830986 24794 raft_consensus.cc:3037] T c6bec901fcec4fb9b6f5477439d16333 P 7eff823b07434c13ad5563cd3601dfa0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:29:18.831632 24794 raft_consensus.cc:2454] T c6bec901fcec4fb9b6f5477439d16333 P 7eff823b07434c13ad5563cd3601dfa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 513ad7723e574f918ed20d18aac3e910 in term 1.
I20260925 07:29:18.831640 24532 raft_consensus.cc:2454] T c6bec901fcec4fb9b6f5477439d16333 P 7c8b76f87ab14a4e94f32e9f8c6d3221 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 513ad7723e574f918ed20d18aac3e910 in term 1.
I20260925 07:29:18.831820 24598 leader_election.cc:302] T c6bec901fcec4fb9b6f5477439d16333 P 513ad7723e574f918ed20d18aac3e910 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 513ad7723e574f918ed20d18aac3e910, 7eff823b07434c13ad5563cd3601dfa0; no voters:
I20260925 07:29:18.831892 24860 raft_consensus.cc:2781] T c6bec901fcec4fb9b6f5477439d16333 P 513ad7723e574f918ed20d18aac3e910 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:29:18.832038 24860 raft_consensus.cc:686] T c6bec901fcec4fb9b6f5477439d16333 P 513ad7723e574f918ed20d18aac3e910 [term 1 LEADER]: Becoming Leader. State: Replica: 513ad7723e574f918ed20d18aac3e910, State: Running, Role: LEADER
I20260925 07:29:18.832119 24860 consensus_queue.cc:237] T c6bec901fcec4fb9b6f5477439d16333 P 513ad7723e574f918ed20d18aac3e910 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eff823b07434c13ad5563cd3601dfa0" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41531 } } peers { permanent_uuid: "7c8b76f87ab14a4e94f32e9f8c6d3221" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 43359 } } peers { permanent_uuid: "513ad7723e574f918ed20d18aac3e910" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 34677 } }
I20260925 07:29:18.832804 24390 catalog_manager.cc:5495] T c6bec901fcec4fb9b6f5477439d16333 P 513ad7723e574f918ed20d18aac3e910 reported cstate change: term changed from 0 to 1, leader changed from <none> to 513ad7723e574f918ed20d18aac3e910 (127.5.158.66). New cstate: current_term: 1 leader_uuid: "513ad7723e574f918ed20d18aac3e910" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eff823b07434c13ad5563cd3601dfa0" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c8b76f87ab14a4e94f32e9f8c6d3221" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 43359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "513ad7723e574f918ed20d18aac3e910" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 34677 } health_report { overall_health: HEALTHY } } }
I20260925 07:29:18.876417 24852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:18.874387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34886 (local address 127.5.158.66:34677)
0925 07:29:18.874437 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:18.874439 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:18.874445 (+ 6us) server_negotiation.cc:407] Connection header received
0925 07:29:18.874502 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:18.874502 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:18.874517 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:18.874567 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 07:29:18.874654 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:18.875137 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:18.875789 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:18.875957 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:18.876001 (+ 44us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:18.876111 (+ 110us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 07:29:18.876204 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 07:29:18.876266 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:18.876307 (+ 41us) server_negotiation.cc:299] Negotiation successful
0925 07:29:18.876313 (+ 6us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260925 07:29:18.878114 24794 raft_consensus.cc:1261] T c6bec901fcec4fb9b6f5477439d16333 P 7eff823b07434c13ad5563cd3601dfa0 [term 1 FOLLOWER]: Refusing update from remote peer 513ad7723e574f918ed20d18aac3e910: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:29:18.878114 24532 raft_consensus.cc:1261] T c6bec901fcec4fb9b6f5477439d16333 P 7c8b76f87ab14a4e94f32e9f8c6d3221 [term 1 FOLLOWER]: Refusing update from remote peer 513ad7723e574f918ed20d18aac3e910: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:29:18.878456 24864 consensus_queue.cc:1035] T c6bec901fcec4fb9b6f5477439d16333 P 513ad7723e574f918ed20d18aac3e910 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c8b76f87ab14a4e94f32e9f8c6d3221" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 43359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:29:18.878582 24860 consensus_queue.cc:1035] T c6bec901fcec4fb9b6f5477439d16333 P 513ad7723e574f918ed20d18aac3e910 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eff823b07434c13ad5563cd3601dfa0" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 41531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:29:18.881264 24868 mvcc.cc:205] Tried to move back new op lower bound from 7333156285962141696 to 7333156285776736256. Current Snapshot: MvccSnapshot[applied={T|T < 7333156285962141696}]
I20260925 07:29:18.882972 24390 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49340:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 07:29:18.883039 24390 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49340:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 07:29:18.883728 24390 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 788c882e23f1442597d942d13b2e8808: Sending DeleteTablet for 3 replicas of tablet c6bec901fcec4fb9b6f5477439d16333
I20260925 07:29:18.884023 24643 tablet_service.cc:1514] Processing DeleteTablet for tablet c6bec901fcec4fb9b6f5477439d16333 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:29:18 UTC) from {username='slave'} at 127.0.0.1:34872
I20260925 07:29:18.884085 24774 tablet_service.cc:1514] Processing DeleteTablet for tablet c6bec901fcec4fb9b6f5477439d16333 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:29:18 UTC) from {username='slave'} at 127.0.0.1:51798
I20260925 07:29:18.884094 24512 tablet_service.cc:1514] Processing DeleteTablet for tablet c6bec901fcec4fb9b6f5477439d16333 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:29:18 UTC) from {username='slave'} at 127.0.0.1:40512
I20260925 07:29:18.884344 24878 tablet_replica.cc:330] T c6bec901fcec4fb9b6f5477439d16333 P 7eff823b07434c13ad5563cd3601dfa0: stopping tablet replica
I20260925 07:29:18.884344 24877 tablet_replica.cc:330] T c6bec901fcec4fb9b6f5477439d16333 P 513ad7723e574f918ed20d18aac3e910: stopping tablet replica
I20260925 07:29:18.884428 24877 raft_consensus.cc:2229] T c6bec901fcec4fb9b6f5477439d16333 P 513ad7723e574f918ed20d18aac3e910 [term 1 LEADER]: Raft consensus shutting down.
I20260925 07:29:18.884488 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 24450
I20260925 07:29:18.885999 24877 raft_consensus.cc:2258] T c6bec901fcec4fb9b6f5477439d16333 P 513ad7723e574f918ed20d18aac3e910 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:29:18.886446 24877 ts_tablet_manager.cc:1905] T c6bec901fcec4fb9b6f5477439d16333 P 513ad7723e574f918ed20d18aac3e910: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:29:18.888301 24877 ts_tablet_manager.cc:1918] T c6bec901fcec4fb9b6f5477439d16333 P 513ad7723e574f918ed20d18aac3e910: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 07:29:18.888413 24877 log.cc:1196] T c6bec901fcec4fb9b6f5477439d16333 P 513ad7723e574f918ed20d18aac3e910: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/ts-1/wal/wals/c6bec901fcec4fb9b6f5477439d16333
I20260925 07:29:18.888702 24877 ts_tablet_manager.cc:1939] T c6bec901fcec4fb9b6f5477439d16333 P 513ad7723e574f918ed20d18aac3e910: Deleting consensus metadata
I20260925 07:29:18.889089 24374 catalog_manager.cc:4841] TS 513ad7723e574f918ed20d18aac3e910 (127.5.158.66:34677): tablet c6bec901fcec4fb9b6f5477439d16333 (table test-table [id=2762e313217f4191b6e54765980e5687]) successfully deleted
I20260925 07:29:18.890146 24878 raft_consensus.cc:2229] T c6bec901fcec4fb9b6f5477439d16333 P 7eff823b07434c13ad5563cd3601dfa0 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260925 07:29:18.890161 24376 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.158.65:43359 (error 108)
I20260925 07:29:18.890321 24878 raft_consensus.cc:2258] T c6bec901fcec4fb9b6f5477439d16333 P 7eff823b07434c13ad5563cd3601dfa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:29:18.890357 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 24581
I20260925 07:29:18.891185 24849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:18.891053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:43359 (local address 127.0.0.1:40528)
0925 07:29:18.891098 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:18.891133 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.65:43359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260925 07:29:18.891203 24878 ts_tablet_manager.cc:1905] T c6bec901fcec4fb9b6f5477439d16333 P 7eff823b07434c13ad5563cd3601dfa0: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 07:29:18.891319 24376 catalog_manager.cc:4584] TS 7c8b76f87ab14a4e94f32e9f8c6d3221 (127.5.158.65:43359): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c6bec901fcec4fb9b6f5477439d16333: Network error: Client connection negotiation failed: client connection to 127.5.158.65:43359: connect: Connection refused (error 111)
I20260925 07:29:18.892401 24878 ts_tablet_manager.cc:1918] T c6bec901fcec4fb9b6f5477439d16333 P 7eff823b07434c13ad5563cd3601dfa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 07:29:18.892467 24878 log.cc:1196] T c6bec901fcec4fb9b6f5477439d16333 P 7eff823b07434c13ad5563cd3601dfa0: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790321110323796-5753-0/minicluster-data/ts-2/wal/wals/c6bec901fcec4fb9b6f5477439d16333
I20260925 07:29:18.892702 24878 ts_tablet_manager.cc:1939] T c6bec901fcec4fb9b6f5477439d16333 P 7eff823b07434c13ad5563cd3601dfa0: Deleting consensus metadata
I20260925 07:29:18.893054 24376 catalog_manager.cc:4841] TS 7eff823b07434c13ad5563cd3601dfa0 (127.5.158.67:41531): tablet c6bec901fcec4fb9b6f5477439d16333 (table test-table [id=2762e313217f4191b6e54765980e5687]) successfully deleted
I20260925 07:29:18.895298 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 24712
I20260925 07:29:18.899643 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 24359
2026-09-25T07:29:18Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (934 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 07:29:18 dist-test-slave-1drq krb5kdc[24883](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 07:29:18 dist-test-slave-1drq krb5kdc[24883](info): set up 2 sockets
Sep 25 07:29:18 dist-test-slave-1drq krb5kdc[24883](info): commencing operation
krb5kdc: starting...
W20260925 07:29:19.848615 5753 mini_kdc.cc:121] Time spent starting KDC: real 0.931s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Sep 25 07:29:19 dist-test-slave-1drq krb5kdc[24883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T07:29:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T07:29:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
I20260925 07:29:19.952116 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:34279
--webserver_interface=127.5.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:45271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.158.126:34279
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:29:20.023012 24896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:29:20.023162 24896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:29:20.023180 24896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:29:20.024108 24896 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:29:20.024137 24896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:29:20.024148 24896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:29:20.024159 24896 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:29:20.024169 24896 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:29:20.025437 24896 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:45271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.158.126:34279
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:34279
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24896
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:29:20.025601 24896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:29:20.025743 24896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:29:20.029881 24901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:20.029983 24896 server_base.cc:1034] running on GCE node
W20260925 07:29:20.029872 24902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:20.029871 24904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:20.030401 24896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:29:20.030678 24896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:29:20.031854 24896 hybrid_clock.cc:648] HybridClock initialized: now 1790321360031834 us; error 33 us; skew 500 ppm
Sep 25 07:29:20 dist-test-slave-1drq krb5kdc[24883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321360, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:29:20.034093 24896 init.cc:377] Logged in from keytab as kudu/127.5.158.126@KRBTEST.COM (short username kudu)
I20260925 07:29:20.034448 24896 webserver.cc:460] Webserver started at http://127.5.158.126:34571/ using document root <none> and password file <none>
I20260925 07:29:20.034614 24896 fs_manager.cc:359] Metadata directory not provided
I20260925 07:29:20.034650 24896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:29:20.034725 24896 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:29:20.035552 24896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/master-0/data/instance:
uuid: "986b8a64b6b64dd28731b5d35a3a7c25"
format_stamp: "Formatted at 2026-09-25 07:29:20 on dist-test-slave-1drq"
server_key: "fbbc0fb98473c5fae881e987b0c43540"
server_key_iv: "69281e71947401ea5e201f75b06475ec"
server_key_version: "encryptionkey@0"
I20260925 07:29:20.035817 24896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/master-0/wal/instance:
uuid: "986b8a64b6b64dd28731b5d35a3a7c25"
format_stamp: "Formatted at 2026-09-25 07:29:20 on dist-test-slave-1drq"
server_key: "fbbc0fb98473c5fae881e987b0c43540"
server_key_iv: "69281e71947401ea5e201f75b06475ec"
server_key_version: "encryptionkey@0"
I20260925 07:29:20.036962 24896 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 07:29:20.037585 24911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:20.037731 24896 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:20.037806 24896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "986b8a64b6b64dd28731b5d35a3a7c25"
format_stamp: "Formatted at 2026-09-25 07:29:20 on dist-test-slave-1drq"
server_key: "fbbc0fb98473c5fae881e987b0c43540"
server_key_iv: "69281e71947401ea5e201f75b06475ec"
server_key_version: "encryptionkey@0"
I20260925 07:29:20.037868 24896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:29:20.049443 24896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:29:20.050979 24896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:29:20.051103 24896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:29:20.054754 24896 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.126:34279
I20260925 07:29:20.054780 24963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.126:34279 every 8 connection(s)
I20260925 07:29:20.055059 24896 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:29:20.055840 24964 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:29:20.055968 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 24896
I20260925 07:29:20.056082 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/master-0/wal/instance
I20260925 07:29:20.056258 5753 external_mini_cluster.cc:1428] Setting key d1962593ae59efd0c2abc3ad9aee1f6a
I20260925 07:29:20.058604 24964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 986b8a64b6b64dd28731b5d35a3a7c25: Bootstrap starting.
I20260925 07:29:20.059142 24964 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 986b8a64b6b64dd28731b5d35a3a7c25: Neither blocks nor log segments found. Creating new log.
I20260925 07:29:20.059442 24964 log.cc:824] T 00000000000000000000000000000000 P 986b8a64b6b64dd28731b5d35a3a7c25: Log is configured to *not* fsync() on all Append() calls
I20260925 07:29:20.060165 24964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 986b8a64b6b64dd28731b5d35a3a7c25: No bootstrap required, opened a new log
Sep 25 07:29:20 dist-test-slave-1drq krb5kdc[24883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:29:20.062605 24964 raft_consensus.cc:348] T 00000000000000000000000000000000 P 986b8a64b6b64dd28731b5d35a3a7c25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "986b8a64b6b64dd28731b5d35a3a7c25" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 34279 } }
I20260925 07:29:20.062745 24964 raft_consensus.cc:374] T 00000000000000000000000000000000 P 986b8a64b6b64dd28731b5d35a3a7c25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:29:20.062803 24964 raft_consensus.cc:729] T 00000000000000000000000000000000 P 986b8a64b6b64dd28731b5d35a3a7c25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 986b8a64b6b64dd28731b5d35a3a7c25, State: Initialized, Role: FOLLOWER
I20260925 07:29:20.062912 24964 consensus_queue.cc:260] T 00000000000000000000000000000000 P 986b8a64b6b64dd28731b5d35a3a7c25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "986b8a64b6b64dd28731b5d35a3a7c25" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 34279 } }
I20260925 07:29:20.062975 24964 raft_consensus.cc:388] T 00000000000000000000000000000000 P 986b8a64b6b64dd28731b5d35a3a7c25 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 07:29:20.062995 24964 raft_consensus.cc:482] T 00000000000000000000000000000000 P 986b8a64b6b64dd28731b5d35a3a7c25 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 07:29:20.063054 24964 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 986b8a64b6b64dd28731b5d35a3a7c25 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:29:20.063763 24964 raft_consensus.cc:504] T 00000000000000000000000000000000 P 986b8a64b6b64dd28731b5d35a3a7c25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "986b8a64b6b64dd28731b5d35a3a7c25" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 34279 } }
I20260925 07:29:20.063869 24964 leader_election.cc:302] T 00000000000000000000000000000000 P 986b8a64b6b64dd28731b5d35a3a7c25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 986b8a64b6b64dd28731b5d35a3a7c25; no voters:
I20260925 07:29:20.064020 24964 leader_election.cc:288] T 00000000000000000000000000000000 P 986b8a64b6b64dd28731b5d35a3a7c25 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 07:29:20.064080 24969 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 986b8a64b6b64dd28731b5d35a3a7c25 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:29:20.064265 24969 raft_consensus.cc:686] T 00000000000000000000000000000000 P 986b8a64b6b64dd28731b5d35a3a7c25 [term 1 LEADER]: Becoming Leader. State: Replica: 986b8a64b6b64dd28731b5d35a3a7c25, State: Running, Role: LEADER
I20260925 07:29:20.064399 24969 consensus_queue.cc:237] T 00000000000000000000000000000000 P 986b8a64b6b64dd28731b5d35a3a7c25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "986b8a64b6b64dd28731b5d35a3a7c25" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 34279 } }
I20260925 07:29:20.064293 24964 sys_catalog.cc:564] T 00000000000000000000000000000000 P 986b8a64b6b64dd28731b5d35a3a7c25 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:29:20.064718 24971 sys_catalog.cc:455] T 00000000000000000000000000000000 P 986b8a64b6b64dd28731b5d35a3a7c25 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "986b8a64b6b64dd28731b5d35a3a7c25" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "986b8a64b6b64dd28731b5d35a3a7c25" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 34279 } } }
I20260925 07:29:20.064726 24970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 986b8a64b6b64dd28731b5d35a3a7c25 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 986b8a64b6b64dd28731b5d35a3a7c25. Latest consensus state: current_term: 1 leader_uuid: "986b8a64b6b64dd28731b5d35a3a7c25" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "986b8a64b6b64dd28731b5d35a3a7c25" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 34279 } } }
I20260925 07:29:20.064805 24971 sys_catalog.cc:458] T 00000000000000000000000000000000 P 986b8a64b6b64dd28731b5d35a3a7c25 [sys.catalog]: This master's current role is: LEADER
I20260925 07:29:20.064810 24970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 986b8a64b6b64dd28731b5d35a3a7c25 [sys.catalog]: This master's current role is: LEADER
I20260925 07:29:20.065009 24973 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 07:29:20.065505 24973 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 07:29:20.066576 24967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:20.056806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51742 (local address 127.5.158.126:34279)
0925 07:29:20.056942 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:20.056948 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:20.056969 (+ 21us) server_negotiation.cc:407] Connection header received
0925 07:29:20.057435 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:20.057441 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:20.057484 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:20.057587 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:20.058346 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:20.059195 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:20.060223 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:20.060617 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:20.060695 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:20.063193 (+ 2498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:20.063197 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:20.063197 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:29:20.063207 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:20.065022 (+ 1815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:20.065462 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:20.065464 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:20.065465 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:20.065573 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:20.065787 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:20.065789 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:20.065789 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:20.066064 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:20.066221 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:20.066341 (+ 120us) server_negotiation.cc:299] Negotiation successful
0925 07:29:20.066444 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":64,"threads_started":1}
I20260925 07:29:20.067379 24973 catalog_manager.cc:1347] Generated new cluster ID: 386f6ae79b6f4ce8af0dbed7a7cb19f7
I20260925 07:29:20.067441 24973 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 07:29:20.074447 24973 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 07:29:20.074846 24973 catalog_manager.cc:1504] Loading token signing keys...
I20260925 07:29:20.086474 24973 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 986b8a64b6b64dd28731b5d35a3a7c25: Generated new TSK 0
I20260925 07:29:20.086634 24973 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.158.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab.
Entry for principal kudu/127.5.158.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab.
I20260925 07:29:20.111732 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.65:0
--local_ip_for_outbound_sockets=127.5.158.65
--webserver_interface=127.5.158.65
--webserver_port=0
--tserver_master_addrs=127.5.158.126:34279
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab
--principal=kudu/127.5.158.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:45271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:29:20.193570 24990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:29:20.193740 24990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:29:20.193761 24990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:29:20.194790 24990 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:29:20.194824 24990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:29:20.194847 24990 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.65
I20260925 07:29:20.196070 24990 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:45271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.65.keytab
--principal=kudu/127.5.158.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.158.65
--webserver_port=0
--tserver_master_addrs=127.5.158.126:34279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24990
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.65
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:29:20.196277 24990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:29:20.196465 24990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 07:29:20.200695 24990 server_base.cc:1034] running on GCE node
W20260925 07:29:20.200659 24995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:20.200675 24998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:20.200667 24996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:20.201025 24990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:29:20.201252 24990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:29:20.202410 24990 hybrid_clock.cc:648] HybridClock initialized: now 1790321360202394 us; error 28 us; skew 500 ppm
Sep 25 07:29:20 dist-test-slave-1drq krb5kdc[24883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321360, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:29:20.204464 24990 init.cc:377] Logged in from keytab as kudu/127.5.158.65@KRBTEST.COM (short username kudu)
I20260925 07:29:20.204823 24990 webserver.cc:460] Webserver started at http://127.5.158.65:40793/ using document root <none> and password file <none>
I20260925 07:29:20.205013 24990 fs_manager.cc:359] Metadata directory not provided
I20260925 07:29:20.205071 24990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:29:20.205219 24990 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:29:20.206127 24990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/ts-0/data/instance:
uuid: "8df33c0046dd482babc20ce5702be453"
format_stamp: "Formatted at 2026-09-25 07:29:20 on dist-test-slave-1drq"
server_key: "43a35a4ae792585dc0d796c588176a13"
server_key_iv: "4b86c3b3587ee8a7d31f33475a75eb9b"
server_key_version: "encryptionkey@0"
I20260925 07:29:20.206426 24990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/ts-0/wal/instance:
uuid: "8df33c0046dd482babc20ce5702be453"
format_stamp: "Formatted at 2026-09-25 07:29:20 on dist-test-slave-1drq"
server_key: "43a35a4ae792585dc0d796c588176a13"
server_key_iv: "4b86c3b3587ee8a7d31f33475a75eb9b"
server_key_version: "encryptionkey@0"
I20260925 07:29:20.207489 24990 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 07:29:20.208051 25005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:20.208177 24990 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:29:20.208258 24990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/ts-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/ts-0/wal
uuid: "8df33c0046dd482babc20ce5702be453"
format_stamp: "Formatted at 2026-09-25 07:29:20 on dist-test-slave-1drq"
server_key: "43a35a4ae792585dc0d796c588176a13"
server_key_iv: "4b86c3b3587ee8a7d31f33475a75eb9b"
server_key_version: "encryptionkey@0"
I20260925 07:29:20.208328 24990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:29:20.231468 24990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:29:20.232970 24990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:29:20.233176 24990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:29:20.233357 24990 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:29:20.233604 24990 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:29:20.233652 24990 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:20.233686 24990 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:29:20.233711 24990 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:20.238916 24990 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.65:36955
I20260925 07:29:20.238966 25118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.65:36955 every 8 connection(s)
I20260925 07:29:20.239223 24990 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
Sep 25 07:29:20 dist-test-slave-1drq krb5kdc[24883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321360, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.65@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:29:20.245903 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 24990
I20260925 07:29:20.246014 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/ts-0/wal/instance
I20260925 07:29:20.246239 5753 external_mini_cluster.cc:1428] Setting key 69897060cdb87277eafdbcefa23d4039
I20260925 07:29:20.246956 24967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:20.239823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:44545 (local address 127.5.158.126:34279)
0925 07:29:20.239861 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:20.239863 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:20.240075 (+ 212us) server_negotiation.cc:407] Connection header received
0925 07:29:20.240501 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:20.240502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:20.240511 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:20.240563 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:20.241396 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:20.241817 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:20.242590 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:20.242748 (+ 158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:20.242790 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:20.244475 (+ 1685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:20.244476 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:20.244477 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:29:20.244482 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:20.245650 (+ 1168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:20.246068 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:20.246069 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:20.246069 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:20.246121 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:20.246374 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:20.246375 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:20.246375 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:20.246498 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:20.246567 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:20.246802 (+ 235us) server_negotiation.cc:299] Negotiation successful
0925 07:29:20.246881 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 07:29:20.247110 25121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:20.239779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:34279 (local address 127.5.158.65:44545)
0925 07:29:20.239932 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:20.239970 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:20.240424 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:20.240629 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:20.240631 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:20.241169 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:29:20.241336 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:20.241339 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:20.241884 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:20.241884 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:20.242526 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:20.242527 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:20.242807 (+ 280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:20.242808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:20.242891 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:20.243248 (+ 357us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:29:20.243255 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:29:20.244402 (+ 1147us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:29:20.245713 (+ 1311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:29:20.245719 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:29:20.245720 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:29:20.245997 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:29:20.246184 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:29:20.246185 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:29:20.246185 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:29:20.246319 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:29:20.246570 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:29:20.246574 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:29:20.246664 (+ 90us) client_negotiation.cc:769] Sending connection context
0925 07:29:20.246770 (+ 106us) client_negotiation.cc:240] Negotiation successful
0925 07:29:20.246911 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":82,"threads_started":1}
I20260925 07:29:20.247483 25119 heartbeater.cc:347] Connected to a master server at 127.5.158.126:34279
I20260925 07:29:20.247576 25119 heartbeater.cc:464] Registering TS with master...
I20260925 07:29:20.247714 25119 heartbeater.cc:511] Master 127.5.158.126:34279 requested a full tablet report, sending...
I20260925 07:29:20.248172 24928 ts_manager.cc:194] Registered new tserver with Master: 8df33c0046dd482babc20ce5702be453 (127.5.158.65:36955)
I20260925 07:29:20.248919 24928 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.65@KRBTEST.COM'} at 127.5.158.65:44545
WARNING: no policy specified for kudu/127.5.158.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab.
Entry for principal kudu/127.5.158.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab.
I20260925 07:29:20.268134 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.66:0
--local_ip_for_outbound_sockets=127.5.158.66
--webserver_interface=127.5.158.66
--webserver_port=0
--tserver_master_addrs=127.5.158.126:34279
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab
--principal=kudu/127.5.158.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:45271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:29:20.342150 25124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:29:20.342307 25124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:29:20.342326 25124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:29:20.343302 25124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:29:20.343333 25124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:29:20.343356 25124 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.66
I20260925 07:29:20.344707 25124 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:45271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.66.keytab
--principal=kudu/127.5.158.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.158.66
--webserver_port=0
--tserver_master_addrs=127.5.158.126:34279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25124
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.66
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:29:20.344866 25124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:29:20.345019 25124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:29:20.349431 25132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:20.349454 25130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:20.349473 25129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:20.349537 25124 server_base.cc:1034] running on GCE node
I20260925 07:29:20.349882 25124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:29:20.350124 25124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:29:20.351274 25124 hybrid_clock.cc:648] HybridClock initialized: now 1790321360351263 us; error 42 us; skew 500 ppm
Sep 25 07:29:20 dist-test-slave-1drq krb5kdc[24883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321360, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:29:20.353503 25124 init.cc:377] Logged in from keytab as kudu/127.5.158.66@KRBTEST.COM (short username kudu)
I20260925 07:29:20.353873 25124 webserver.cc:460] Webserver started at http://127.5.158.66:40563/ using document root <none> and password file <none>
I20260925 07:29:20.354079 25124 fs_manager.cc:359] Metadata directory not provided
I20260925 07:29:20.354138 25124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:29:20.354275 25124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:29:20.355223 25124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/ts-1/data/instance:
uuid: "f0cf14dd70794558a451bcf587964e82"
format_stamp: "Formatted at 2026-09-25 07:29:20 on dist-test-slave-1drq"
server_key: "0bcd054a82a939b677fe7c4ee753b414"
server_key_iv: "05feb88b9ddb030d79ebfec58395aed0"
server_key_version: "encryptionkey@0"
I20260925 07:29:20.355562 25124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/ts-1/wal/instance:
uuid: "f0cf14dd70794558a451bcf587964e82"
format_stamp: "Formatted at 2026-09-25 07:29:20 on dist-test-slave-1drq"
server_key: "0bcd054a82a939b677fe7c4ee753b414"
server_key_iv: "05feb88b9ddb030d79ebfec58395aed0"
server_key_version: "encryptionkey@0"
I20260925 07:29:20.356771 25124 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 07:29:20.357471 25139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:20.357630 25124 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:20.357713 25124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/ts-1/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/ts-1/wal
uuid: "f0cf14dd70794558a451bcf587964e82"
format_stamp: "Formatted at 2026-09-25 07:29:20 on dist-test-slave-1drq"
server_key: "0bcd054a82a939b677fe7c4ee753b414"
server_key_iv: "05feb88b9ddb030d79ebfec58395aed0"
server_key_version: "encryptionkey@0"
I20260925 07:29:20.357785 25124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:29:20.391215 25124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:29:20.393245 25124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:29:20.393405 25124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:29:20.393600 25124 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:29:20.393965 25124 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:29:20.394033 25124 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:20.394066 25124 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:29:20.394097 25124 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:20.399461 25124 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.66:44337
I20260925 07:29:20.399549 25252 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.66:44337 every 8 connection(s)
I20260925 07:29:20.399832 25124 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
I20260925 07:29:20.402482 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 25124
I20260925 07:29:20.402602 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/ts-1/wal/instance
I20260925 07:29:20.402784 5753 external_mini_cluster.cc:1428] Setting key 21e72f60a883139c5dd45664cd799e3e
Sep 25 07:29:20 dist-test-slave-1drq krb5kdc[24883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321360, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.66@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:29:20.409638 24967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:20.400456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:52691 (local address 127.5.158.126:34279)
0925 07:29:20.400529 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:20.400531 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:20.400684 (+ 153us) server_negotiation.cc:407] Connection header received
0925 07:29:20.401283 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:20.401284 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:20.401294 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:20.401341 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:20.402133 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:20.402754 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:20.404135 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:20.404312 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:20.404370 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:20.406712 (+ 2342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:20.406714 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:20.406714 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:29:20.406722 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:20.408168 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:20.408611 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:20.408612 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:20.408612 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:20.408674 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:20.408934 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:20.408935 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:20.408936 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:20.409058 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:20.409178 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:20.409442 (+ 264us) server_negotiation.cc:299] Negotiation successful
0925 07:29:20.409524 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 07:29:20.409809 25255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:20.400413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:34279 (local address 127.5.158.66:52691)
0925 07:29:20.400572 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:20.400602 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:20.401194 (+ 592us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:20.401395 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:20.401398 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:20.401897 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:29:20.402065 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:20.402069 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:20.402851 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:20.402852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:20.404055 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:20.404059 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:20.404391 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:20.404392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:20.404529 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:20.405061 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:29:20.405070 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:29:20.406597 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:29:20.408275 (+ 1678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:29:20.408282 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:29:20.408284 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:29:20.408515 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:29:20.408745 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:29:20.408745 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:29:20.408746 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:29:20.408856 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:29:20.409169 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:29:20.409176 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:29:20.409307 (+ 131us) client_negotiation.cc:769] Sending connection context
0925 07:29:20.409418 (+ 111us) client_negotiation.cc:240] Negotiation successful
0925 07:29:20.409588 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":94,"threads_started":1}
I20260925 07:29:20.410236 25253 heartbeater.cc:347] Connected to a master server at 127.5.158.126:34279
I20260925 07:29:20.410326 25253 heartbeater.cc:464] Registering TS with master...
I20260925 07:29:20.410507 25253 heartbeater.cc:511] Master 127.5.158.126:34279 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.5.158.67@KRBTEST.COM; defaulting to no policy
I20260925 07:29:20.410950 24928 ts_manager.cc:194] Registered new tserver with Master: f0cf14dd70794558a451bcf587964e82 (127.5.158.66:44337)
I20260925 07:29:20.411337 24928 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.66@KRBTEST.COM'} at 127.5.158.66:52691
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab.
Entry for principal kudu/127.5.158.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab.
I20260925 07:29:20.426047 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.67:0
--local_ip_for_outbound_sockets=127.5.158.67
--webserver_interface=127.5.158.67
--webserver_port=0
--tserver_master_addrs=127.5.158.126:34279
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab
--principal=kudu/127.5.158.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:45271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:29:20.496281 25258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:29:20.496440 25258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:29:20.496461 25258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:29:20.497424 25258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:29:20.497454 25258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:29:20.497478 25258 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.67
I20260925 07:29:20.498710 25258 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:45271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.67.keytab
--principal=kudu/127.5.158.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.158.67
--webserver_port=0
--tserver_master_addrs=127.5.158.126:34279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25258
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.67
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:29:20.498853 25258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:29:20.498992 25258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:29:20.503647 25266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:20.503710 25258 server_base.cc:1034] running on GCE node
W20260925 07:29:20.503633 25264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:20.503625 25263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:20.504035 25258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:29:20.504248 25258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:29:20.505378 25258 hybrid_clock.cc:648] HybridClock initialized: now 1790321360505367 us; error 27 us; skew 500 ppm
Sep 25 07:29:20 dist-test-slave-1drq krb5kdc[24883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321360, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:29:20.507673 25258 init.cc:377] Logged in from keytab as kudu/127.5.158.67@KRBTEST.COM (short username kudu)
I20260925 07:29:20.508000 25258 webserver.cc:460] Webserver started at http://127.5.158.67:44029/ using document root <none> and password file <none>
I20260925 07:29:20.508189 25258 fs_manager.cc:359] Metadata directory not provided
I20260925 07:29:20.508226 25258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:29:20.508307 25258 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:29:20.509289 25258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/ts-2/data/instance:
uuid: "29917538a9bd424dad4e78d4dad41f39"
format_stamp: "Formatted at 2026-09-25 07:29:20 on dist-test-slave-1drq"
server_key: "fc6cee53d4b2003c18c72e915f3e6405"
server_key_iv: "0e4c0da2aa4e089b66a5e2551d31ec09"
server_key_version: "encryptionkey@0"
I20260925 07:29:20.509588 25258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/ts-2/wal/instance:
uuid: "29917538a9bd424dad4e78d4dad41f39"
format_stamp: "Formatted at 2026-09-25 07:29:20 on dist-test-slave-1drq"
server_key: "fc6cee53d4b2003c18c72e915f3e6405"
server_key_iv: "0e4c0da2aa4e089b66a5e2551d31ec09"
server_key_version: "encryptionkey@0"
I20260925 07:29:20.510768 25258 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 07:29:20.511521 25273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:20.511693 25258 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 07:29:20.511786 25258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/ts-2/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/ts-2/wal
uuid: "29917538a9bd424dad4e78d4dad41f39"
format_stamp: "Formatted at 2026-09-25 07:29:20 on dist-test-slave-1drq"
server_key: "fc6cee53d4b2003c18c72e915f3e6405"
server_key_iv: "0e4c0da2aa4e089b66a5e2551d31ec09"
server_key_version: "encryptionkey@0"
I20260925 07:29:20.511864 25258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:29:20.523341 25258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:29:20.525460 25258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:29:20.525622 25258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:29:20.525820 25258 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:29:20.526081 25258 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:29:20.526130 25258 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:20.526165 25258 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:29:20.526207 25258 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:20.531418 25258 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.67:37535
I20260925 07:29:20.531467 25386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.67:37535 every 8 connection(s)
I20260925 07:29:20.531746 25258 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
Sep 25 07:29:20 dist-test-slave-1drq krb5kdc[24883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321360, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.67@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:29:20.539670 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 25258
I20260925 07:29:20.539815 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/ts-2/wal/instance
I20260925 07:29:20.540010 5753 external_mini_cluster.cc:1428] Setting key d646c479fe982a1632ed04bb75144e2f
I20260925 07:29:20.540553 24967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:20.532397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:43827 (local address 127.5.158.126:34279)
0925 07:29:20.532456 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:20.532458 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:20.532635 (+ 177us) server_negotiation.cc:407] Connection header received
0925 07:29:20.533147 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:20.533148 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:20.533161 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:20.533228 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:20.534002 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:20.534457 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:20.535360 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:20.535544 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:20.535619 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:20.537401 (+ 1782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:20.537402 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:20.537402 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:29:20.537410 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:20.539054 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:20.539462 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:20.539464 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:20.539464 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:20.539524 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:20.539811 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:20.539812 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:20.539813 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:20.539954 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:20.540037 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:20.540382 (+ 345us) server_negotiation.cc:299] Negotiation successful
0925 07:29:20.540465 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260925 07:29:20.540802 25389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:20.532361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:34279 (local address 127.5.158.67:43827)
0925 07:29:20.532516 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:20.532558 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:20.533010 (+ 452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:20.533287 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:20.533290 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:20.533755 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:29:20.533906 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:20.533909 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:20.534541 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:20.534542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:20.535276 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:20.535278 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:20.535626 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:20.535627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:20.535754 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:20.536134 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:29:20.536141 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:29:20.537312 (+ 1171us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:29:20.539160 (+ 1848us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:29:20.539167 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:29:20.539169 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:29:20.539382 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:29:20.539595 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:29:20.539596 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:29:20.539596 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:29:20.539719 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:29:20.540040 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:29:20.540047 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:29:20.540229 (+ 182us) client_negotiation.cc:769] Sending connection context
0925 07:29:20.540344 (+ 115us) client_negotiation.cc:240] Negotiation successful
0925 07:29:20.540493 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":70,"threads_started":1}
I20260925 07:29:20.541204 25387 heartbeater.cc:347] Connected to a master server at 127.5.158.126:34279
I20260925 07:29:20.541294 25387 heartbeater.cc:464] Registering TS with master...
I20260925 07:29:20.541469 25387 heartbeater.cc:511] Master 127.5.158.126:34279 requested a full tablet report, sending...
I20260925 07:29:20.541788 24928 ts_manager.cc:194] Registered new tserver with Master: 29917538a9bd424dad4e78d4dad41f39 (127.5.158.67:37535)
I20260925 07:29:20.542156 24928 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.158.67@KRBTEST.COM'} at 127.5.158.67:43827
I20260925 07:29:20.551687 5753 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 07:29:20.557953 24967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:20.552877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51744 (local address 127.5.158.126:34279)
0925 07:29:20.552923 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:20.552925 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:20.553037 (+ 112us) server_negotiation.cc:407] Connection header received
0925 07:29:20.553120 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:20.553121 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:20.553130 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:20.553175 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:20.553669 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:20.554146 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:20.554839 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:20.555020 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:20.555084 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:20.556033 (+ 949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:20.556034 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:20.556034 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:29:20.556040 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:20.556909 (+ 869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:20.557279 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:20.557280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:20.557281 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:20.557331 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:20.557501 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:20.557502 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:20.557502 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:20.557626 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:20.557720 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:20.557804 (+ 84us) server_negotiation.cc:299] Negotiation successful
0925 07:29:20.557873 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260925 07:29:20.559294 24928 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51744:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 07:29:20.561033 24928 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 07:29:20.566939 25395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:20.563123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:36955 (local address 127.0.0.1:36208)
0925 07:29:20.563387 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:20.563408 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:20.563466 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:20.563851 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:20.563852 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:20.563853 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:29:20.563883 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:20.563884 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:20.564996 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:20.564996 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:20.565789 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:20.565792 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:20.566681 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:20.566682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:20.566749 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:20.566758 (+ 9us) client_negotiation.cc:769] Sending connection context
0925 07:29:20.566831 (+ 73us) client_negotiation.cc:240] Negotiation successful
0925 07:29:20.566878 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"mutex_wait_us":28,"thread_start_us":53,"threads_started":1}
I20260925 07:29:20.566946 25397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:20.563233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36208 (local address 127.5.158.65:36955)
0925 07:29:20.563432 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:20.563436 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:20.563446 (+ 10us) server_negotiation.cc:407] Connection header received
0925 07:29:20.563616 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:20.563619 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:20.563634 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:20.563712 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:29:20.563956 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:20.564909 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:20.565927 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:20.566530 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:20.566594 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:20.566650 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:20.566848 (+ 198us) server_negotiation.cc:299] Negotiation successful
0925 07:29:20.566884 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":75,"threads_started":1}
I20260925 07:29:20.567157 25396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:20.563121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.67:37535 (local address 127.0.0.1:53218)
0925 07:29:20.563231 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:20.563267 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:20.563582 (+ 315us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:20.563807 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:20.563808 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:20.563810 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:29:20.563864 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:20.563865 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:20.564868 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:20.564869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:20.566064 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:20.566068 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:20.566908 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:20.566909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:20.566963 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:20.566966 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 07:29:20.567026 (+ 60us) client_negotiation.cc:240] Negotiation successful
0925 07:29:20.567071 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85,"mutex_wait_us":46,"spinlock_wait_cycles":768,"thread_start_us":63,"threads_started":1}
I20260925 07:29:20.567339 25398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:20.563303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.66:44337 (local address 127.0.0.1:41510)
0925 07:29:20.563500 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:20.563508 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:20.563545 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:20.563724 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:20.563727 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:20.563729 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:29:20.563782 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:20.563784 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:20.564993 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:20.564994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:20.566132 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:20.566134 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:20.566922 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:20.566922 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:20.566963 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:20.566969 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 07:29:20.567247 (+ 278us) client_negotiation.cc:240] Negotiation successful
0925 07:29:20.567285 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":33,"thread_start_us":36,"threads_started":1}
I20260925 07:29:20.567363 25399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:20.563272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53218 (local address 127.5.158.67:37535)
0925 07:29:20.563450 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:20.563455 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:20.563465 (+ 10us) server_negotiation.cc:407] Connection header received
0925 07:29:20.563667 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:20.563669 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:20.563682 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:20.563744 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:29:20.563964 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:20.564660 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:20.566160 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:20.566754 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:20.566823 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:20.566873 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:20.567263 (+ 390us) server_negotiation.cc:299] Negotiation successful
0925 07:29:20.567299 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":80,"threads_started":1}
I20260925 07:29:20.567616 25400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:20.563441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41510 (local address 127.5.158.66:44337)
0925 07:29:20.563564 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:20.563567 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:20.563578 (+ 11us) server_negotiation.cc:407] Connection header received
0925 07:29:20.563612 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:20.563614 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:20.563629 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:20.563721 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:29:20.563837 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:20.564878 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:20.566213 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:20.566801 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:20.566854 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:20.566896 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:20.567499 (+ 603us) server_negotiation.cc:299] Negotiation successful
0925 07:29:20.567547 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":70,"threads_started":1}
I20260925 07:29:20.567780 25053 tablet_service.cc:1467] Processing CreateTablet for tablet dafa8a10a0f0468a9efed200769666af (DEFAULT_TABLE table=test-table [id=6251a629213e4a17be795323dd448218]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:29:20.567885 25321 tablet_service.cc:1467] Processing CreateTablet for tablet dafa8a10a0f0468a9efed200769666af (DEFAULT_TABLE table=test-table [id=6251a629213e4a17be795323dd448218]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:29:20.568122 25053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dafa8a10a0f0468a9efed200769666af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:29:20.568094 25187 tablet_service.cc:1467] Processing CreateTablet for tablet dafa8a10a0f0468a9efed200769666af (DEFAULT_TABLE table=test-table [id=6251a629213e4a17be795323dd448218]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:29:20.568185 25321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dafa8a10a0f0468a9efed200769666af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:29:20.568352 25187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dafa8a10a0f0468a9efed200769666af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:29:20.570649 25401 tablet_bootstrap.cc:492] T dafa8a10a0f0468a9efed200769666af P 8df33c0046dd482babc20ce5702be453: Bootstrap starting.
I20260925 07:29:20.571332 25401 tablet_bootstrap.cc:654] T dafa8a10a0f0468a9efed200769666af P 8df33c0046dd482babc20ce5702be453: Neither blocks nor log segments found. Creating new log.
I20260925 07:29:20.571594 25403 tablet_bootstrap.cc:492] T dafa8a10a0f0468a9efed200769666af P f0cf14dd70794558a451bcf587964e82: Bootstrap starting.
I20260925 07:29:20.571911 25401 log.cc:824] T dafa8a10a0f0468a9efed200769666af P 8df33c0046dd482babc20ce5702be453: Log is configured to *not* fsync() on all Append() calls
I20260925 07:29:20.572278 25403 tablet_bootstrap.cc:654] T dafa8a10a0f0468a9efed200769666af P f0cf14dd70794558a451bcf587964e82: Neither blocks nor log segments found. Creating new log.
I20260925 07:29:20.572702 25401 tablet_bootstrap.cc:492] T dafa8a10a0f0468a9efed200769666af P 8df33c0046dd482babc20ce5702be453: No bootstrap required, opened a new log
I20260925 07:29:20.572772 25401 ts_tablet_manager.cc:1397] T dafa8a10a0f0468a9efed200769666af P 8df33c0046dd482babc20ce5702be453: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 07:29:20.572814 25403 log.cc:824] T dafa8a10a0f0468a9efed200769666af P f0cf14dd70794558a451bcf587964e82: Log is configured to *not* fsync() on all Append() calls
I20260925 07:29:20.573145 25402 tablet_bootstrap.cc:492] T dafa8a10a0f0468a9efed200769666af P 29917538a9bd424dad4e78d4dad41f39: Bootstrap starting.
I20260925 07:29:20.573751 25402 tablet_bootstrap.cc:654] T dafa8a10a0f0468a9efed200769666af P 29917538a9bd424dad4e78d4dad41f39: Neither blocks nor log segments found. Creating new log.
I20260925 07:29:20.574236 25402 log.cc:824] T dafa8a10a0f0468a9efed200769666af P 29917538a9bd424dad4e78d4dad41f39: Log is configured to *not* fsync() on all Append() calls
I20260925 07:29:20.574688 25403 tablet_bootstrap.cc:492] T dafa8a10a0f0468a9efed200769666af P f0cf14dd70794558a451bcf587964e82: No bootstrap required, opened a new log
I20260925 07:29:20.574755 25403 ts_tablet_manager.cc:1397] T dafa8a10a0f0468a9efed200769666af P f0cf14dd70794558a451bcf587964e82: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260925 07:29:20.574718 25401 raft_consensus.cc:348] T dafa8a10a0f0468a9efed200769666af P 8df33c0046dd482babc20ce5702be453 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29917538a9bd424dad4e78d4dad41f39" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 37535 } } peers { permanent_uuid: "8df33c0046dd482babc20ce5702be453" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 36955 } } peers { permanent_uuid: "f0cf14dd70794558a451bcf587964e82" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 44337 } }
I20260925 07:29:20.574875 25401 raft_consensus.cc:374] T dafa8a10a0f0468a9efed200769666af P 8df33c0046dd482babc20ce5702be453 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:29:20.574913 25401 raft_consensus.cc:729] T dafa8a10a0f0468a9efed200769666af P 8df33c0046dd482babc20ce5702be453 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df33c0046dd482babc20ce5702be453, State: Initialized, Role: FOLLOWER
I20260925 07:29:20.575017 25401 consensus_queue.cc:260] T dafa8a10a0f0468a9efed200769666af P 8df33c0046dd482babc20ce5702be453 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29917538a9bd424dad4e78d4dad41f39" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 37535 } } peers { permanent_uuid: "8df33c0046dd482babc20ce5702be453" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 36955 } } peers { permanent_uuid: "f0cf14dd70794558a451bcf587964e82" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 44337 } }
I20260925 07:29:20.575244 25401 ts_tablet_manager.cc:1428] T dafa8a10a0f0468a9efed200769666af P 8df33c0046dd482babc20ce5702be453: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 07:29:20.575296 25119 heartbeater.cc:503] Master 127.5.158.126:34279 was elected leader, sending a full tablet report...
I20260925 07:29:20.576180 25403 raft_consensus.cc:348] T dafa8a10a0f0468a9efed200769666af P f0cf14dd70794558a451bcf587964e82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29917538a9bd424dad4e78d4dad41f39" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 37535 } } peers { permanent_uuid: "8df33c0046dd482babc20ce5702be453" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 36955 } } peers { permanent_uuid: "f0cf14dd70794558a451bcf587964e82" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 44337 } }
I20260925 07:29:20.576318 25403 raft_consensus.cc:374] T dafa8a10a0f0468a9efed200769666af P f0cf14dd70794558a451bcf587964e82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:29:20.576361 25403 raft_consensus.cc:729] T dafa8a10a0f0468a9efed200769666af P f0cf14dd70794558a451bcf587964e82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0cf14dd70794558a451bcf587964e82, State: Initialized, Role: FOLLOWER
I20260925 07:29:20.576462 25403 consensus_queue.cc:260] T dafa8a10a0f0468a9efed200769666af P f0cf14dd70794558a451bcf587964e82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29917538a9bd424dad4e78d4dad41f39" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 37535 } } peers { permanent_uuid: "8df33c0046dd482babc20ce5702be453" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 36955 } } peers { permanent_uuid: "f0cf14dd70794558a451bcf587964e82" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 44337 } }
I20260925 07:29:20.576679 25403 ts_tablet_manager.cc:1428] T dafa8a10a0f0468a9efed200769666af P f0cf14dd70794558a451bcf587964e82: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 07:29:20.576704 25253 heartbeater.cc:503] Master 127.5.158.126:34279 was elected leader, sending a full tablet report...
I20260925 07:29:20.577062 25402 tablet_bootstrap.cc:492] T dafa8a10a0f0468a9efed200769666af P 29917538a9bd424dad4e78d4dad41f39: No bootstrap required, opened a new log
I20260925 07:29:20.577150 25402 ts_tablet_manager.cc:1397] T dafa8a10a0f0468a9efed200769666af P 29917538a9bd424dad4e78d4dad41f39: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260925 07:29:20.578469 25402 raft_consensus.cc:348] T dafa8a10a0f0468a9efed200769666af P 29917538a9bd424dad4e78d4dad41f39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29917538a9bd424dad4e78d4dad41f39" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 37535 } } peers { permanent_uuid: "8df33c0046dd482babc20ce5702be453" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 36955 } } peers { permanent_uuid: "f0cf14dd70794558a451bcf587964e82" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 44337 } }
I20260925 07:29:20.578606 25402 raft_consensus.cc:374] T dafa8a10a0f0468a9efed200769666af P 29917538a9bd424dad4e78d4dad41f39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:29:20.578648 25402 raft_consensus.cc:729] T dafa8a10a0f0468a9efed200769666af P 29917538a9bd424dad4e78d4dad41f39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29917538a9bd424dad4e78d4dad41f39, State: Initialized, Role: FOLLOWER
I20260925 07:29:20.578773 25402 consensus_queue.cc:260] T dafa8a10a0f0468a9efed200769666af P 29917538a9bd424dad4e78d4dad41f39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29917538a9bd424dad4e78d4dad41f39" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 37535 } } peers { permanent_uuid: "8df33c0046dd482babc20ce5702be453" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 36955 } } peers { permanent_uuid: "f0cf14dd70794558a451bcf587964e82" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 44337 } }
I20260925 07:29:20.579010 25402 ts_tablet_manager.cc:1428] T dafa8a10a0f0468a9efed200769666af P 29917538a9bd424dad4e78d4dad41f39: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 07:29:20.579072 25387 heartbeater.cc:503] Master 127.5.158.126:34279 was elected leader, sending a full tablet report...
W20260925 07:29:20.650346 25254 tablet.cc:2369] T dafa8a10a0f0468a9efed200769666af P f0cf14dd70794558a451bcf587964e82: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 07:29:20.664666 25407 raft_consensus.cc:482] T dafa8a10a0f0468a9efed200769666af P 8df33c0046dd482babc20ce5702be453 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 07:29:20.664899 25407 raft_consensus.cc:504] T dafa8a10a0f0468a9efed200769666af P 8df33c0046dd482babc20ce5702be453 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29917538a9bd424dad4e78d4dad41f39" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 37535 } } peers { permanent_uuid: "8df33c0046dd482babc20ce5702be453" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 36955 } } peers { permanent_uuid: "f0cf14dd70794558a451bcf587964e82" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 44337 } }
I20260925 07:29:20.665280 25407 leader_election.cc:288] T dafa8a10a0f0468a9efed200769666af P 8df33c0046dd482babc20ce5702be453 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29917538a9bd424dad4e78d4dad41f39 (127.5.158.67:37535), f0cf14dd70794558a451bcf587964e82 (127.5.158.66:44337)
I20260925 07:29:20.667752 25400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:20.665501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:55519 (local address 127.5.158.66:44337)
0925 07:29:20.665534 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:20.665536 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:20.665664 (+ 128us) server_negotiation.cc:407] Connection header received
0925 07:29:20.665695 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:20.665695 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:20.665709 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:20.665751 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:29:20.665881 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:20.666321 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:20.667122 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:20.667449 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:20.667522 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:20.667529 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:20.667656 (+ 127us) server_negotiation.cc:299] Negotiation successful
0925 07:29:20.667690 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260925 07:29:20.667755 25399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:20.665483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:56439 (local address 127.5.158.67:37535)
0925 07:29:20.665527 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:20.665529 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:20.665537 (+ 8us) server_negotiation.cc:407] Connection header received
0925 07:29:20.665564 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:20.665565 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:20.665572 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:20.665622 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:29:20.665765 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:20.666304 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:20.667121 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:20.667447 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:20.667521 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:20.667529 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:20.667670 (+ 141us) server_negotiation.cc:299] Negotiation successful
0925 07:29:20.667703 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260925 07:29:20.667755 25121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:20.665407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.67:37535 (local address 127.5.158.65:56439)
0925 07:29:20.665471 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:20.665482 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:20.665516 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:20.665628 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:20.665629 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:20.665630 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:29:20.665687 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:20.665689 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:20.666374 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:20.666375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:20.667040 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:20.667042 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:20.667525 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:20.667525 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:20.667596 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:20.667602 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 07:29:20.667656 (+ 54us) client_negotiation.cc:240] Negotiation successful
0925 07:29:20.667692 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":26}
I20260925 07:29:20.667762 25410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:20.665438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.66:44337 (local address 127.5.158.65:55519)
0925 07:29:20.665604 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:20.665611 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:20.665656 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:20.665752 (+ 96us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:20.665752 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:20.665753 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:29:20.665821 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:20.665822 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:20.666391 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:20.666392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:20.667039 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:20.667041 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:20.667522 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:20.667523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:20.667591 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:20.667594 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 07:29:20.667650 (+ 56us) client_negotiation.cc:240] Negotiation successful
0925 07:29:20.667684 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":3,"thread_start_us":121,"threads_started":1}
I20260925 07:29:20.668044 25341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dafa8a10a0f0468a9efed200769666af" candidate_uuid: "8df33c0046dd482babc20ce5702be453" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29917538a9bd424dad4e78d4dad41f39" is_pre_election: true
I20260925 07:29:20.668077 25207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dafa8a10a0f0468a9efed200769666af" candidate_uuid: "8df33c0046dd482babc20ce5702be453" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0cf14dd70794558a451bcf587964e82" is_pre_election: true
I20260925 07:29:20.668165 25207 raft_consensus.cc:2454] T dafa8a10a0f0468a9efed200769666af P f0cf14dd70794558a451bcf587964e82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8df33c0046dd482babc20ce5702be453 in term 0.
I20260925 07:29:20.668166 25341 raft_consensus.cc:2454] T dafa8a10a0f0468a9efed200769666af P 29917538a9bd424dad4e78d4dad41f39 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8df33c0046dd482babc20ce5702be453 in term 0.
I20260925 07:29:20.668354 25006 leader_election.cc:302] T dafa8a10a0f0468a9efed200769666af P 8df33c0046dd482babc20ce5702be453 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8df33c0046dd482babc20ce5702be453, f0cf14dd70794558a451bcf587964e82; no voters:
I20260925 07:29:20.668486 25407 raft_consensus.cc:2781] T dafa8a10a0f0468a9efed200769666af P 8df33c0046dd482babc20ce5702be453 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 07:29:20.668534 25407 raft_consensus.cc:482] T dafa8a10a0f0468a9efed200769666af P 8df33c0046dd482babc20ce5702be453 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 07:29:20.668556 25407 raft_consensus.cc:3037] T dafa8a10a0f0468a9efed200769666af P 8df33c0046dd482babc20ce5702be453 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:29:20.669222 25407 raft_consensus.cc:504] T dafa8a10a0f0468a9efed200769666af P 8df33c0046dd482babc20ce5702be453 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29917538a9bd424dad4e78d4dad41f39" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 37535 } } peers { permanent_uuid: "8df33c0046dd482babc20ce5702be453" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 36955 } } peers { permanent_uuid: "f0cf14dd70794558a451bcf587964e82" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 44337 } }
I20260925 07:29:20.669339 25407 leader_election.cc:288] T dafa8a10a0f0468a9efed200769666af P 8df33c0046dd482babc20ce5702be453 [CANDIDATE]: Term 1 election: Requested vote from peers 29917538a9bd424dad4e78d4dad41f39 (127.5.158.67:37535), f0cf14dd70794558a451bcf587964e82 (127.5.158.66:44337)
I20260925 07:29:20.669449 25341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dafa8a10a0f0468a9efed200769666af" candidate_uuid: "8df33c0046dd482babc20ce5702be453" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29917538a9bd424dad4e78d4dad41f39"
I20260925 07:29:20.669453 25207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dafa8a10a0f0468a9efed200769666af" candidate_uuid: "8df33c0046dd482babc20ce5702be453" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0cf14dd70794558a451bcf587964e82"
I20260925 07:29:20.669538 25207 raft_consensus.cc:3037] T dafa8a10a0f0468a9efed200769666af P f0cf14dd70794558a451bcf587964e82 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:29:20.669534 25341 raft_consensus.cc:3037] T dafa8a10a0f0468a9efed200769666af P 29917538a9bd424dad4e78d4dad41f39 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:29:20.670394 25341 raft_consensus.cc:2454] T dafa8a10a0f0468a9efed200769666af P 29917538a9bd424dad4e78d4dad41f39 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df33c0046dd482babc20ce5702be453 in term 1.
I20260925 07:29:20.670394 25207 raft_consensus.cc:2454] T dafa8a10a0f0468a9efed200769666af P f0cf14dd70794558a451bcf587964e82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df33c0046dd482babc20ce5702be453 in term 1.
I20260925 07:29:20.670552 25006 leader_election.cc:302] T dafa8a10a0f0468a9efed200769666af P 8df33c0046dd482babc20ce5702be453 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8df33c0046dd482babc20ce5702be453, f0cf14dd70794558a451bcf587964e82; no voters:
I20260925 07:29:20.670656 25407 raft_consensus.cc:2781] T dafa8a10a0f0468a9efed200769666af P 8df33c0046dd482babc20ce5702be453 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:29:20.670815 25407 raft_consensus.cc:686] T dafa8a10a0f0468a9efed200769666af P 8df33c0046dd482babc20ce5702be453 [term 1 LEADER]: Becoming Leader. State: Replica: 8df33c0046dd482babc20ce5702be453, State: Running, Role: LEADER
I20260925 07:29:20.670913 25407 consensus_queue.cc:237] T dafa8a10a0f0468a9efed200769666af P 8df33c0046dd482babc20ce5702be453 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29917538a9bd424dad4e78d4dad41f39" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 37535 } } peers { permanent_uuid: "8df33c0046dd482babc20ce5702be453" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 36955 } } peers { permanent_uuid: "f0cf14dd70794558a451bcf587964e82" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 44337 } }
I20260925 07:29:20.671686 24927 catalog_manager.cc:5495] T dafa8a10a0f0468a9efed200769666af P 8df33c0046dd482babc20ce5702be453 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8df33c0046dd482babc20ce5702be453 (127.5.158.65). New cstate: current_term: 1 leader_uuid: "8df33c0046dd482babc20ce5702be453" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29917538a9bd424dad4e78d4dad41f39" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 37535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8df33c0046dd482babc20ce5702be453" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 36955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f0cf14dd70794558a451bcf587964e82" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 44337 } health_report { overall_health: UNKNOWN } } }
I20260925 07:29:20.684116 25397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:20.681639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36224 (local address 127.5.158.65:36955)
0925 07:29:20.681708 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:20.681710 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:20.681748 (+ 38us) server_negotiation.cc:407] Connection header received
0925 07:29:20.681843 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:20.681844 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:20.681866 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:20.681945 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 07:29:20.682065 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:20.682434 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:20.683271 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:20.683459 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:20.683556 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:20.683753 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 07:29:20.683845 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 07:29:20.683935 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:20.683998 (+ 63us) server_negotiation.cc:299] Negotiation successful
0925 07:29:20.684017 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260925 07:29:20.685756 25207 raft_consensus.cc:1261] T dafa8a10a0f0468a9efed200769666af P f0cf14dd70794558a451bcf587964e82 [term 1 FOLLOWER]: Refusing update from remote peer 8df33c0046dd482babc20ce5702be453: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:29:20.685756 25341 raft_consensus.cc:1261] T dafa8a10a0f0468a9efed200769666af P 29917538a9bd424dad4e78d4dad41f39 [term 1 FOLLOWER]: Refusing update from remote peer 8df33c0046dd482babc20ce5702be453: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:29:20.686038 25407 consensus_queue.cc:1035] T dafa8a10a0f0468a9efed200769666af P 8df33c0046dd482babc20ce5702be453 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0cf14dd70794558a451bcf587964e82" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 44337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:29:20.686149 25411 consensus_queue.cc:1035] T dafa8a10a0f0468a9efed200769666af P 8df33c0046dd482babc20ce5702be453 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29917538a9bd424dad4e78d4dad41f39" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 37535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:29:20.688225 25414 mvcc.cc:205] Tried to move back new op lower bound from 7333156293366419456 to 7333156293308653568. Current Snapshot: MvccSnapshot[applied={T|T < 7333156293366419456}]
I20260925 07:29:20.688962 25416 mvcc.cc:205] Tried to move back new op lower bound from 7333156293366419456 to 7333156293308653568. Current Snapshot: MvccSnapshot[applied={T|T < 7333156293366419456}]
I20260925 07:29:20.690282 24928 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51744:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 07:29:20.690376 24928 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51744:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 07:29:20.691002 24928 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 986b8a64b6b64dd28731b5d35a3a7c25: Sending DeleteTablet for 3 replicas of tablet dafa8a10a0f0468a9efed200769666af
I20260925 07:29:20.691304 25321 tablet_service.cc:1514] Processing DeleteTablet for tablet dafa8a10a0f0468a9efed200769666af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:29:20 UTC) from {username='kudu', principal='kudu/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:53218
I20260925 07:29:20.691310 25053 tablet_service.cc:1514] Processing DeleteTablet for tablet dafa8a10a0f0468a9efed200769666af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:29:20 UTC) from {username='kudu', principal='kudu/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:36208
I20260925 07:29:20.691367 25187 tablet_service.cc:1514] Processing DeleteTablet for tablet dafa8a10a0f0468a9efed200769666af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:29:20 UTC) from {username='kudu', principal='kudu/127.5.158.126@KRBTEST.COM'} at 127.0.0.1:41510
I20260925 07:29:20.691538 25426 tablet_replica.cc:330] T dafa8a10a0f0468a9efed200769666af P f0cf14dd70794558a451bcf587964e82: stopping tablet replica
I20260925 07:29:20.691547 25425 tablet_replica.cc:330] T dafa8a10a0f0468a9efed200769666af P 8df33c0046dd482babc20ce5702be453: stopping tablet replica
I20260925 07:29:20.691622 25426 raft_consensus.cc:2229] T dafa8a10a0f0468a9efed200769666af P f0cf14dd70794558a451bcf587964e82 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 07:29:20.691627 25425 raft_consensus.cc:2229] T dafa8a10a0f0468a9efed200769666af P 8df33c0046dd482babc20ce5702be453 [term 1 LEADER]: Raft consensus shutting down.
I20260925 07:29:20.691705 25426 raft_consensus.cc:2258] T dafa8a10a0f0468a9efed200769666af P f0cf14dd70794558a451bcf587964e82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:29:20.691694 25425 raft_consensus.cc:2258] T dafa8a10a0f0468a9efed200769666af P 8df33c0046dd482babc20ce5702be453 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:29:20.691916 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 24990
I20260925 07:29:20.692768 25426 ts_tablet_manager.cc:1905] T dafa8a10a0f0468a9efed200769666af P f0cf14dd70794558a451bcf587964e82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:29:20.694389 25426 ts_tablet_manager.cc:1918] T dafa8a10a0f0468a9efed200769666af P f0cf14dd70794558a451bcf587964e82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 07:29:20.694456 25426 log.cc:1196] T dafa8a10a0f0468a9efed200769666af P f0cf14dd70794558a451bcf587964e82: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/ts-1/wal/wals/dafa8a10a0f0468a9efed200769666af
I20260925 07:29:20.694693 25426 ts_tablet_manager.cc:1939] T dafa8a10a0f0468a9efed200769666af P f0cf14dd70794558a451bcf587964e82: Deleting consensus metadata
I20260925 07:29:20.695196 24912 catalog_manager.cc:4841] TS f0cf14dd70794558a451bcf587964e82 (127.5.158.66:44337): tablet dafa8a10a0f0468a9efed200769666af (table test-table [id=6251a629213e4a17be795323dd448218]) successfully deleted
W20260925 07:29:20.696810 24912 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.158.65:36955 (error 108)
I20260925 07:29:20.697247 25398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:20.697119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:36955 (local address 127.0.0.1:36226)
0925 07:29:20.697164 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:20.697213 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.65:36955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260925 07:29:20.697290 25424 tablet_replica.cc:330] T dafa8a10a0f0468a9efed200769666af P 29917538a9bd424dad4e78d4dad41f39: stopping tablet replica
I20260925 07:29:20.697304 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 25124
I20260925 07:29:20.697371 25424 raft_consensus.cc:2229] T dafa8a10a0f0468a9efed200769666af P 29917538a9bd424dad4e78d4dad41f39 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 07:29:20.697453 25424 raft_consensus.cc:2258] T dafa8a10a0f0468a9efed200769666af P 29917538a9bd424dad4e78d4dad41f39 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:29:20.697803 25424 ts_tablet_manager.cc:1905] T dafa8a10a0f0468a9efed200769666af P 29917538a9bd424dad4e78d4dad41f39: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 07:29:20.698937 24912 catalog_manager.cc:4584] TS 8df33c0046dd482babc20ce5702be453 (127.5.158.65:36955): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet dafa8a10a0f0468a9efed200769666af: Network error: Client connection negotiation failed: client connection to 127.5.158.65:36955: connect: Connection refused (error 111)
I20260925 07:29:20.699321 25424 ts_tablet_manager.cc:1918] T dafa8a10a0f0468a9efed200769666af P 29917538a9bd424dad4e78d4dad41f39: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 07:29:20.699396 25424 log.cc:1196] T dafa8a10a0f0468a9efed200769666af P 29917538a9bd424dad4e78d4dad41f39: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790321110323796-5753-0/minicluster-data/ts-2/wal/wals/dafa8a10a0f0468a9efed200769666af
I20260925 07:29:20.699632 25424 ts_tablet_manager.cc:1939] T dafa8a10a0f0468a9efed200769666af P 29917538a9bd424dad4e78d4dad41f39: Deleting consensus metadata
I20260925 07:29:20.700104 24915 catalog_manager.cc:4841] TS 29917538a9bd424dad4e78d4dad41f39 (127.5.158.67:37535): tablet dafa8a10a0f0468a9efed200769666af (table test-table [id=6251a629213e4a17be795323dd448218]) successfully deleted
I20260925 07:29:20.702318 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 25258
I20260925 07:29:20.707123 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 24896
2026-09-25T07:29:20Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (1808 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-25T07:29:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T07:29:20Z Disabled control of system clock
I20260925 07:29:20.732833 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:37757
--webserver_interface=127.5.158.126
--webserver_port=0
--builtin_ntp_servers=127.5.158.84:41057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.158.126:37757
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 07:29:20.804873 25431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:29:20.805049 25431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:29:20.805068 25431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:29:20.806219 25431 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:29:20.806264 25431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:29:20.806278 25431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:29:20.806293 25431 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:29:20.806353 25431 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:29:20.807662 25431 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:41057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.158.126:37757
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:37757
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25431
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:29:20.807825 25431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:29:20.808095 25431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:29:20.812419 25437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:20.812417 25436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:20.812419 25439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:20.812593 25431 server_base.cc:1034] running on GCE node
I20260925 07:29:20.812853 25431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:29:20.813189 25431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:29:20.814333 25431 hybrid_clock.cc:648] HybridClock initialized: now 1790321360814324 us; error 25 us; skew 500 ppm
I20260925 07:29:20.815423 25431 webserver.cc:460] Webserver started at http://127.5.158.126:36583/ using document root <none> and password file <none>
I20260925 07:29:20.815649 25431 fs_manager.cc:359] Metadata directory not provided
I20260925 07:29:20.815693 25431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:29:20.815824 25431 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:29:20.816766 25431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/master-0/data/instance:
uuid: "5f0df05593b3442a8ba8f8226e1b06b9"
format_stamp: "Formatted at 2026-09-25 07:29:20 on dist-test-slave-1drq"
server_key: "3a4dfa77c15c7604652f8d5b719c20ed"
server_key_iv: "0ef17f4028dbc751c8eaa01f4e3573f3"
server_key_version: "encryptionkey@0"
I20260925 07:29:20.817135 25431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/master-0/wal/instance:
uuid: "5f0df05593b3442a8ba8f8226e1b06b9"
format_stamp: "Formatted at 2026-09-25 07:29:20 on dist-test-slave-1drq"
server_key: "3a4dfa77c15c7604652f8d5b719c20ed"
server_key_iv: "0ef17f4028dbc751c8eaa01f4e3573f3"
server_key_version: "encryptionkey@0"
I20260925 07:29:20.818262 25431 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 07:29:20.818895 25445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:20.819080 25431 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 07:29:20.819162 25431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "5f0df05593b3442a8ba8f8226e1b06b9"
format_stamp: "Formatted at 2026-09-25 07:29:20 on dist-test-slave-1drq"
server_key: "3a4dfa77c15c7604652f8d5b719c20ed"
server_key_iv: "0ef17f4028dbc751c8eaa01f4e3573f3"
server_key_version: "encryptionkey@0"
I20260925 07:29:20.819226 25431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:29:20.826941 25431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:29:20.827168 25431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:29:20.827301 25431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:29:20.831065 25431 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.126:37757
I20260925 07:29:20.831089 25497 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.126:37757 every 8 connection(s)
I20260925 07:29:20.831372 25431 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:29:20.832139 25498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:29:20.833943 25498 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f0df05593b3442a8ba8f8226e1b06b9: Bootstrap starting.
I20260925 07:29:20.834365 25498 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5f0df05593b3442a8ba8f8226e1b06b9: Neither blocks nor log segments found. Creating new log.
I20260925 07:29:20.834561 25498 log.cc:824] T 00000000000000000000000000000000 P 5f0df05593b3442a8ba8f8226e1b06b9: Log is configured to *not* fsync() on all Append() calls
I20260925 07:29:20.835122 25498 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f0df05593b3442a8ba8f8226e1b06b9: No bootstrap required, opened a new log
I20260925 07:29:20.836427 25498 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f0df05593b3442a8ba8f8226e1b06b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f0df05593b3442a8ba8f8226e1b06b9" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37757 } }
I20260925 07:29:20.836490 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 25431
I20260925 07:29:20.836561 25498 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5f0df05593b3442a8ba8f8226e1b06b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:29:20.836604 25498 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f0df05593b3442a8ba8f8226e1b06b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f0df05593b3442a8ba8f8226e1b06b9, State: Initialized, Role: FOLLOWER
I20260925 07:29:20.836627 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/master-0/wal/instance
I20260925 07:29:20.836764 25498 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f0df05593b3442a8ba8f8226e1b06b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f0df05593b3442a8ba8f8226e1b06b9" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37757 } }
I20260925 07:29:20.836808 5753 external_mini_cluster.cc:1428] Setting key 1067d05deb765c2e4f05a7715bb60ac7
I20260925 07:29:20.836854 25498 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5f0df05593b3442a8ba8f8226e1b06b9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 07:29:20.836894 25498 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f0df05593b3442a8ba8f8226e1b06b9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 07:29:20.836936 25498 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f0df05593b3442a8ba8f8226e1b06b9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:29:20.837543 25498 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f0df05593b3442a8ba8f8226e1b06b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f0df05593b3442a8ba8f8226e1b06b9" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37757 } }
I20260925 07:29:20.837632 25498 leader_election.cc:302] T 00000000000000000000000000000000 P 5f0df05593b3442a8ba8f8226e1b06b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5f0df05593b3442a8ba8f8226e1b06b9; no voters:
I20260925 07:29:20.837759 25498 leader_election.cc:288] T 00000000000000000000000000000000 P 5f0df05593b3442a8ba8f8226e1b06b9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 07:29:20.837822 25503 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f0df05593b3442a8ba8f8226e1b06b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:29:20.838011 25498 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f0df05593b3442a8ba8f8226e1b06b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:29:20.837984 25503 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5f0df05593b3442a8ba8f8226e1b06b9 [term 1 LEADER]: Becoming Leader. State: Replica: 5f0df05593b3442a8ba8f8226e1b06b9, State: Running, Role: LEADER
I20260925 07:29:20.838132 25503 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f0df05593b3442a8ba8f8226e1b06b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f0df05593b3442a8ba8f8226e1b06b9" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37757 } }
I20260925 07:29:20.839251 25504 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f0df05593b3442a8ba8f8226e1b06b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f0df05593b3442a8ba8f8226e1b06b9. Latest consensus state: current_term: 1 leader_uuid: "5f0df05593b3442a8ba8f8226e1b06b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f0df05593b3442a8ba8f8226e1b06b9" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37757 } } }
I20260925 07:29:20.839344 25504 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f0df05593b3442a8ba8f8226e1b06b9 [sys.catalog]: This master's current role is: LEADER
W20260925 07:29:20.839725 25518 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5f0df05593b3442a8ba8f8226e1b06b9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260925 07:29:20.839488 25505 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f0df05593b3442a8ba8f8226e1b06b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5f0df05593b3442a8ba8f8226e1b06b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f0df05593b3442a8ba8f8226e1b06b9" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 37757 } } }
W20260925 07:29:20.839793 25518 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 07:29:20.839803 25505 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f0df05593b3442a8ba8f8226e1b06b9 [sys.catalog]: This master's current role is: LEADER
I20260925 07:29:20.839922 25519 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 07:29:20.840114 25519 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 07:29:20.841657 25519 catalog_manager.cc:1347] Generated new cluster ID: a56c8f5616fc437bb417262d81e3ea80
I20260925 07:29:20.841707 25519 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 07:29:20.843204 25502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:20.837247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57444 (local address 127.5.158.126:37757)
0925 07:29:20.837473 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:20.837480 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:20.837508 (+ 28us) server_negotiation.cc:407] Connection header received
0925 07:29:20.837601 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:20.837607 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:20.837676 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:20.837813 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:20.838481 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:20.840257 (+ 1776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:20.841256 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:20.841793 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:20.841911 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:20.842503 (+ 592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:20.842507 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:20.842508 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:29:20.842682 (+ 174us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:20.842730 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:29:20.842744 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:20.842816 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:20.842887 (+ 71us) server_negotiation.cc:299] Negotiation successful
0925 07:29:20.843034 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":112,"threads_started":1}
I20260925 07:29:20.851716 25519 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 07:29:20.852272 25519 catalog_manager.cc:1504] Loading token signing keys...
I20260925 07:29:20.855868 25519 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5f0df05593b3442a8ba8f8226e1b06b9: Generated new TSK 0
I20260925 07:29:20.856002 25519 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 07:29:20.864440 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.65:0
--local_ip_for_outbound_sockets=127.5.158.65
--webserver_interface=127.5.158.65
--webserver_port=0
--tserver_master_addrs=127.5.158.126:37757
--builtin_ntp_servers=127.5.158.84:41057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 07:29:20.940197 25522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:29:20.940457 25522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:29:20.940495 25522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:29:20.941686 25522 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:29:20.941752 25522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:29:20.941799 25522 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.65
I20260925 07:29:20.943168 25522 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:41057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.158.65
--webserver_port=0
--tserver_master_addrs=127.5.158.126:37757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25522
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.65
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:29:20.943444 25522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:29:20.943747 25522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:29:20.949527 25527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:20.949519 25528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:20.949553 25530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:20.949980 25522 server_base.cc:1034] running on GCE node
I20260925 07:29:20.950168 25522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:29:20.950397 25522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:29:20.951556 25522 hybrid_clock.cc:648] HybridClock initialized: now 1790321360951537 us; error 31 us; skew 500 ppm
I20260925 07:29:20.952952 25522 webserver.cc:460] Webserver started at http://127.5.158.65:34273/ using document root <none> and password file <none>
I20260925 07:29:20.953202 25522 fs_manager.cc:359] Metadata directory not provided
I20260925 07:29:20.953259 25522 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:29:20.953375 25522 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:29:20.954634 25522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/ts-0/data/instance:
uuid: "aef05e0706bc4268a182fe5938ba6e51"
format_stamp: "Formatted at 2026-09-25 07:29:20 on dist-test-slave-1drq"
server_key: "ac0bdbe8c3aec41e92b2858b7b539db8"
server_key_iv: "b72045f0a4be8bb2ee11acf9325e6066"
server_key_version: "encryptionkey@0"
I20260925 07:29:20.955039 25522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/ts-0/wal/instance:
uuid: "aef05e0706bc4268a182fe5938ba6e51"
format_stamp: "Formatted at 2026-09-25 07:29:20 on dist-test-slave-1drq"
server_key: "ac0bdbe8c3aec41e92b2858b7b539db8"
server_key_iv: "b72045f0a4be8bb2ee11acf9325e6066"
server_key_version: "encryptionkey@0"
I20260925 07:29:20.956560 25522 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 07:29:20.957279 25536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:20.957454 25522 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 07:29:20.957552 25522 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/ts-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/ts-0/wal
uuid: "aef05e0706bc4268a182fe5938ba6e51"
format_stamp: "Formatted at 2026-09-25 07:29:20 on dist-test-slave-1drq"
server_key: "ac0bdbe8c3aec41e92b2858b7b539db8"
server_key_iv: "b72045f0a4be8bb2ee11acf9325e6066"
server_key_version: "encryptionkey@0"
I20260925 07:29:20.957633 25522 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:29:20.976049 25522 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:29:20.976325 25522 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:29:20.976475 25522 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:29:20.976693 25522 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:29:20.977039 25522 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:29:20.977123 25522 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:20.977164 25522 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:29:20.977191 25522 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:20.983086 25522 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.65:34025
I20260925 07:29:20.983466 25522 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/ts-0/data/info.pb
I20260925 07:29:20.989308 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 25522
I20260925 07:29:20.989420 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/ts-0/wal/instance
I20260925 07:29:20.989631 5753 external_mini_cluster.cc:1428] Setting key 8621f1c2e984ee34b898afa15179b792
I20260925 07:29:20.990087 25502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:20.984440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.65:55247 (local address 127.5.158.126:37757)
0925 07:29:20.984483 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:20.984485 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:20.984495 (+ 10us) server_negotiation.cc:407] Connection header received
0925 07:29:20.985061 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:20.985062 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:20.985075 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:20.985164 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:20.985642 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:20.986444 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:20.987851 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:20.988164 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:20.988201 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:20.989782 (+ 1581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:20.989783 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:20.989784 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:29:20.989789 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:20.989804 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:29:20.989810 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:20.989840 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:20.989979 (+ 139us) server_negotiation.cc:299] Negotiation successful
0925 07:29:20.990011 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 07:29:20.989938 25649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.65:34025 every 8 connection(s)
I20260925 07:29:20.990484 25652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:20.984156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37757 (local address 127.5.158.65:55247)
0925 07:29:20.984352 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:20.984392 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:20.984949 (+ 557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:20.985288 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:20.985292 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:20.985301 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:29:20.985547 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:20.985551 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:20.986548 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:20.986549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:20.987740 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:20.987743 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:20.988284 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:20.988284 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:20.988423 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:20.989589 (+ 1166us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 07:29:20.989589 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:29:20.989627 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 07:29:20.989655 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:29:20.989884 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:29:20.989889 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:29:20.989892 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 07:29:20.990197 (+ 305us) client_negotiation.cc:240] Negotiation successful
0925 07:29:20.990293 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":88,"threads_started":1}
I20260925 07:29:20.991695 25650 heartbeater.cc:347] Connected to a master server at 127.5.158.126:37757
I20260925 07:29:20.991775 25650 heartbeater.cc:464] Registering TS with master...
I20260925 07:29:20.991920 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.66:0
--local_ip_for_outbound_sockets=127.5.158.66
--webserver_interface=127.5.158.66
--webserver_port=0
--tserver_master_addrs=127.5.158.126:37757
--builtin_ntp_servers=127.5.158.84:41057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 07:29:20.991963 25650 heartbeater.cc:511] Master 127.5.158.126:37757 requested a full tablet report, sending...
I20260925 07:29:20.993637 25462 ts_manager.cc:194] Registered new tserver with Master: aef05e0706bc4268a182fe5938ba6e51 (127.5.158.65:34025)
I20260925 07:29:20.994225 25462 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.158.65:55247
W20260925 07:29:21.066278 25653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:29:21.066486 25653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:29:21.066541 25653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:29:21.067531 25653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:29:21.067585 25653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:29:21.067623 25653 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.66
I20260925 07:29:21.068773 25653 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:41057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.158.66
--webserver_port=0
--tserver_master_addrs=127.5.158.126:37757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25653
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.66
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:29:21.068996 25653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:29:21.069286 25653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:29:21.073956 25658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:21.074039 25661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:21.073988 25659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:21.074108 25653 server_base.cc:1034] running on GCE node
I20260925 07:29:21.074349 25653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:29:21.074565 25653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:29:21.075697 25653 hybrid_clock.cc:648] HybridClock initialized: now 1790321361075687 us; error 45 us; skew 500 ppm
I20260925 07:29:21.076814 25653 webserver.cc:460] Webserver started at http://127.5.158.66:36653/ using document root <none> and password file <none>
I20260925 07:29:21.077023 25653 fs_manager.cc:359] Metadata directory not provided
I20260925 07:29:21.077062 25653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:29:21.077232 25653 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:29:21.078163 25653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/ts-1/data/instance:
uuid: "6f62732bcd7f4ff99e9bafb6664615a5"
format_stamp: "Formatted at 2026-09-25 07:29:21 on dist-test-slave-1drq"
server_key: "9cb33fd07641df755c4759f68ecc97c3"
server_key_iv: "5d76dd4c42f661a9b677f0654129c0bd"
server_key_version: "encryptionkey@0"
I20260925 07:29:21.078501 25653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/ts-1/wal/instance:
uuid: "6f62732bcd7f4ff99e9bafb6664615a5"
format_stamp: "Formatted at 2026-09-25 07:29:21 on dist-test-slave-1drq"
server_key: "9cb33fd07641df755c4759f68ecc97c3"
server_key_iv: "5d76dd4c42f661a9b677f0654129c0bd"
server_key_version: "encryptionkey@0"
I20260925 07:29:21.079648 25653 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 07:29:21.080296 25667 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:21.080457 25653 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:29:21.080549 25653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/ts-1/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/ts-1/wal
uuid: "6f62732bcd7f4ff99e9bafb6664615a5"
format_stamp: "Formatted at 2026-09-25 07:29:21 on dist-test-slave-1drq"
server_key: "9cb33fd07641df755c4759f68ecc97c3"
server_key_iv: "5d76dd4c42f661a9b677f0654129c0bd"
server_key_version: "encryptionkey@0"
I20260925 07:29:21.080624 25653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:29:21.094291 25653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:29:21.094512 25653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:29:21.094637 25653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:29:21.094820 25653 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:29:21.095082 25653 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:29:21.095129 25653 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:21.095160 25653 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:29:21.095182 25653 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:21.100746 25653 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.66:42889
I20260925 07:29:21.100783 25780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.66:42889 every 8 connection(s)
I20260925 07:29:21.101059 25653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/ts-1/data/info.pb
I20260925 07:29:21.106104 25502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:21.101780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:57839 (local address 127.5.158.126:37757)
0925 07:29:21.101829 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:21.101831 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:21.102063 (+ 232us) server_negotiation.cc:407] Connection header received
0925 07:29:21.102520 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:21.102521 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:21.102553 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:21.102613 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:21.103243 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:21.103745 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:21.104703 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:21.104906 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:21.104981 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:21.105810 (+ 829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:21.105811 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:21.105812 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:29:21.105816 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:21.105838 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:29:21.105847 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:21.105908 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:21.106000 (+ 92us) server_negotiation.cc:299] Negotiation successful
0925 07:29:21.106028 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 07:29:21.106308 25783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:21.101770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37757 (local address 127.5.158.66:57839)
0925 07:29:21.101949 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:21.101975 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:21.102409 (+ 434us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:21.102672 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:21.102675 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:21.102685 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:29:21.103145 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:21.103148 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:21.103847 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:21.103849 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:21.104603 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:21.104606 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:21.105006 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:21.105008 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:21.105142 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:21.105668 (+ 526us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 07:29:21.105669 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:29:21.105692 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 07:29:21.105712 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:29:21.105908 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:29:21.105914 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:29:21.105917 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 07:29:21.106007 (+ 90us) client_negotiation.cc:240] Negotiation successful
0925 07:29:21.106097 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":78,"threads_started":1}
I20260925 07:29:21.106786 25781 heartbeater.cc:347] Connected to a master server at 127.5.158.126:37757
I20260925 07:29:21.106855 25781 heartbeater.cc:464] Registering TS with master...
I20260925 07:29:21.107048 25781 heartbeater.cc:511] Master 127.5.158.126:37757 requested a full tablet report, sending...
I20260925 07:29:21.107152 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 25653
I20260925 07:29:21.107287 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/ts-1/wal/instance
I20260925 07:29:21.107434 25462 ts_manager.cc:194] Registered new tserver with Master: 6f62732bcd7f4ff99e9bafb6664615a5 (127.5.158.66:42889)
I20260925 07:29:21.107512 5753 external_mini_cluster.cc:1428] Setting key b69915fa5c6bf55f766d73dca4e6bde9
I20260925 07:29:21.107970 25462 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.158.66:57839
I20260925 07:29:21.108430 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.158.67:0
--local_ip_for_outbound_sockets=127.5.158.67
--webserver_interface=127.5.158.67
--webserver_port=0
--tserver_master_addrs=127.5.158.126:37757
--builtin_ntp_servers=127.5.158.84:41057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 07:29:21.180691 25784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:29:21.180910 25784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:29:21.180945 25784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:29:21.182240 25784 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:29:21.182331 25784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:29:21.182377 25784 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.158.67
I20260925 07:29:21.183677 25784 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:41057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.158.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.158.67
--webserver_port=0
--tserver_master_addrs=127.5.158.126:37757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25784
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.158.67
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:29:21.183895 25784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:29:21.184142 25784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 07:29:21.188759 25784 server_base.cc:1034] running on GCE node
W20260925 07:29:21.188702 25789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:21.188725 25792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:21.188712 25790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:21.189121 25784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:29:21.189342 25784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:29:21.190505 25784 hybrid_clock.cc:648] HybridClock initialized: now 1790321361190494 us; error 26 us; skew 500 ppm
I20260925 07:29:21.191881 25784 webserver.cc:460] Webserver started at http://127.5.158.67:43849/ using document root <none> and password file <none>
I20260925 07:29:21.192094 25784 fs_manager.cc:359] Metadata directory not provided
I20260925 07:29:21.192139 25784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:29:21.192245 25784 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:29:21.193475 25784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/ts-2/data/instance:
uuid: "c129e37176e24862896138306355891c"
format_stamp: "Formatted at 2026-09-25 07:29:21 on dist-test-slave-1drq"
server_key: "7c5b9555d71705e514a0ec61cfd63336"
server_key_iv: "6262dd5d17886fdea95f5fb2d92e3e75"
server_key_version: "encryptionkey@0"
I20260925 07:29:21.193845 25784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/ts-2/wal/instance:
uuid: "c129e37176e24862896138306355891c"
format_stamp: "Formatted at 2026-09-25 07:29:21 on dist-test-slave-1drq"
server_key: "7c5b9555d71705e514a0ec61cfd63336"
server_key_iv: "6262dd5d17886fdea95f5fb2d92e3e75"
server_key_version: "encryptionkey@0"
I20260925 07:29:21.195035 25784 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 07:29:21.195624 25798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:21.195771 25784 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:29:21.195854 25784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/ts-2/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/ts-2/wal
uuid: "c129e37176e24862896138306355891c"
format_stamp: "Formatted at 2026-09-25 07:29:21 on dist-test-slave-1drq"
server_key: "7c5b9555d71705e514a0ec61cfd63336"
server_key_iv: "6262dd5d17886fdea95f5fb2d92e3e75"
server_key_version: "encryptionkey@0"
I20260925 07:29:21.195930 25784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:29:21.225056 25784 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:29:21.225312 25784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:29:21.225459 25784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:29:21.225662 25784 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 07:29:21.225932 25784 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 07:29:21.225979 25784 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:21.226011 25784 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 07:29:21.226032 25784 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:21.231240 25784 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.67:40073
I20260925 07:29:21.231281 25911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.67:40073 every 8 connection(s)
I20260925 07:29:21.231519 25784 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/ts-2/data/info.pb
I20260925 07:29:21.232424 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 25784
I20260925 07:29:21.232520 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/ts-2/wal/instance
I20260925 07:29:21.232702 5753 external_mini_cluster.cc:1428] Setting key 5671bf7ffd3d2fcf3e8ac64be5fc191c
I20260925 07:29:21.235706 25502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:21.232061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.67:49799 (local address 127.5.158.126:37757)
0925 07:29:21.232096 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:21.232098 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:21.232267 (+ 169us) server_negotiation.cc:407] Connection header received
0925 07:29:21.232715 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:21.232716 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:21.232728 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:21.232794 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:21.233480 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:21.233938 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:21.234692 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:21.234847 (+ 155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:21.234891 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:21.235547 (+ 656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:21.235547 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:21.235548 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:29:21.235551 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:21.235562 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:29:21.235565 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:21.235600 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:21.235640 (+ 40us) server_negotiation.cc:299] Negotiation successful
0925 07:29:21.235659 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260925 07:29:21.235920 25914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:21.232036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:37757 (local address 127.5.158.67:49799)
0925 07:29:21.232178 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:21.232209 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:21.232626 (+ 417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:21.232872 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:21.232876 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:21.232884 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:29:21.233402 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:21.233406 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:21.234002 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:21.234003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:21.234632 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:21.234633 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:21.234907 (+ 274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:21.234907 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:21.234991 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:21.235429 (+ 438us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 07:29:21.235430 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:29:21.235462 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 07:29:21.235480 (+ 18us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:29:21.235600 (+ 120us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:29:21.235602 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:29:21.235604 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 07:29:21.235641 (+ 37us) client_negotiation.cc:240] Negotiation successful
0925 07:29:21.235700 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":70,"threads_started":1}
I20260925 07:29:21.236256 25912 heartbeater.cc:347] Connected to a master server at 127.5.158.126:37757
I20260925 07:29:21.236326 25912 heartbeater.cc:464] Registering TS with master...
I20260925 07:29:21.236483 25912 heartbeater.cc:511] Master 127.5.158.126:37757 requested a full tablet report, sending...
I20260925 07:29:21.236758 25461 ts_manager.cc:194] Registered new tserver with Master: c129e37176e24862896138306355891c (127.5.158.67:40073)
I20260925 07:29:21.237128 25461 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.158.67:49799
I20260925 07:29:21.244230 5753 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 07:29:21.247931 25502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:21.245444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57456 (local address 127.5.158.126:37757)
0925 07:29:21.245511 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:21.245512 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:21.245591 (+ 79us) server_negotiation.cc:407] Connection header received
0925 07:29:21.245648 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:21.245649 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:21.245658 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:21.245717 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:21.245881 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:21.246306 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:21.246950 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:21.247110 (+ 160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:21.247229 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:21.247711 (+ 482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:21.247712 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:21.247712 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:29:21.247716 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:21.247727 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:29:21.247730 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:21.247793 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:21.247840 (+ 47us) server_negotiation.cc:299] Negotiation successful
0925 07:29:21.247859 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 07:29:21.249238 25461 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57456:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 07:29:21.250787 25461 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 07:29:21.256116 25922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:21.252663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56908 (local address 127.5.158.67:40073)
0925 07:29:21.252795 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:21.252799 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:21.252808 (+ 9us) server_negotiation.cc:407] Connection header received
0925 07:29:21.252839 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:21.252841 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:21.252855 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:21.252931 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:29:21.253128 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:21.253986 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:21.255281 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:21.255782 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:21.255863 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:21.255934 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:21.256023 (+ 89us) server_negotiation.cc:299] Negotiation successful
0925 07:29:21.256052 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":74,"threads_started":1}
I20260925 07:29:21.256109 25920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:21.252513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.67:40073 (local address 127.0.0.1:56908)
0925 07:29:21.252628 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:21.252661 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:21.252721 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:21.252991 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:21.252993 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:21.252996 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:29:21.253048 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:21.253050 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:21.254053 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:21.254054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:21.255136 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:21.255138 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:21.255868 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:21.255869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:21.255934 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:21.255945 (+ 11us) client_negotiation.cc:769] Sending connection context
0925 07:29:21.256001 (+ 56us) client_negotiation.cc:240] Negotiation successful
0925 07:29:21.256053 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":73,"threads_started":1}
I20260925 07:29:21.256242 25921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:21.252551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.66:42889 (local address 127.0.0.1:41316)
0925 07:29:21.252899 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:21.252908 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:21.252951 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:21.253147 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:21.253148 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:21.253150 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:29:21.253188 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:21.253190 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:21.254323 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:21.254324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:21.255269 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:21.255271 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:21.256046 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:21.256047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:21.256106 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:21.256109 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 07:29:21.256133 (+ 24us) client_negotiation.cc:240] Negotiation successful
0925 07:29:21.256167 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":73,"threads_started":1}
I20260925 07:29:21.256453 25925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:21.252880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44388 (local address 127.5.158.65:34025)
0925 07:29:21.253051 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:21.253055 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:21.253367 (+ 312us) server_negotiation.cc:407] Connection header received
0925 07:29:21.253465 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:21.253468 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:21.253481 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:21.253526 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:29:21.253703 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:21.254586 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:21.255487 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:21.256062 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:21.256110 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:21.256179 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:21.256333 (+ 154us) server_negotiation.cc:299] Negotiation successful
0925 07:29:21.256368 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":78,"threads_started":1}
I20260925 07:29:21.256522 25924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:21.252807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41316 (local address 127.5.158.66:42889)
0925 07:29:21.252989 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:21.252992 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:21.253000 (+ 8us) server_negotiation.cc:407] Connection header received
0925 07:29:21.253049 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:21.253050 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:21.253065 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:21.253150 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:29:21.253251 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:21.254239 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:21.255358 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:21.255955 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:21.256306 (+ 351us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:21.256381 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:21.256403 (+ 22us) server_negotiation.cc:299] Negotiation successful
0925 07:29:21.256435 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":87,"threads_started":1}
I20260925 07:29:21.256383 25923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:21.252711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:34025 (local address 127.0.0.1:44388)
0925 07:29:21.253292 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:21.253318 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:21.253391 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:21.253597 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:21.253597 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:21.253599 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:29:21.253636 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:21.253637 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:21.254659 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:21.254660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:21.255392 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:21.255394 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:21.256206 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:21.256206 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:21.256252 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:21.256255 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 07:29:21.256306 (+ 51us) client_negotiation.cc:240] Negotiation successful
0925 07:29:21.256335 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":64,"threads_started":1}
I20260925 07:29:21.257067 25715 tablet_service.cc:1467] Processing CreateTablet for tablet d9e82a9592b741e5a90fabd5c4c3481e (DEFAULT_TABLE table=test-table [id=595d124d1b9940e7acdc2f2f88de262a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:29:21.257164 25846 tablet_service.cc:1467] Processing CreateTablet for tablet d9e82a9592b741e5a90fabd5c4c3481e (DEFAULT_TABLE table=test-table [id=595d124d1b9940e7acdc2f2f88de262a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:29:21.257237 25584 tablet_service.cc:1467] Processing CreateTablet for tablet d9e82a9592b741e5a90fabd5c4c3481e (DEFAULT_TABLE table=test-table [id=595d124d1b9940e7acdc2f2f88de262a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 07:29:21.257380 25715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9e82a9592b741e5a90fabd5c4c3481e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:29:21.257450 25846 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9e82a9592b741e5a90fabd5c4c3481e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:29:21.257478 25584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9e82a9592b741e5a90fabd5c4c3481e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:29:21.259636 25926 tablet_bootstrap.cc:492] T d9e82a9592b741e5a90fabd5c4c3481e P 6f62732bcd7f4ff99e9bafb6664615a5: Bootstrap starting.
I20260925 07:29:21.260370 25927 tablet_bootstrap.cc:492] T d9e82a9592b741e5a90fabd5c4c3481e P c129e37176e24862896138306355891c: Bootstrap starting.
I20260925 07:29:21.260573 25928 tablet_bootstrap.cc:492] T d9e82a9592b741e5a90fabd5c4c3481e P aef05e0706bc4268a182fe5938ba6e51: Bootstrap starting.
I20260925 07:29:21.260674 25926 tablet_bootstrap.cc:654] T d9e82a9592b741e5a90fabd5c4c3481e P 6f62732bcd7f4ff99e9bafb6664615a5: Neither blocks nor log segments found. Creating new log.
I20260925 07:29:21.261041 25927 tablet_bootstrap.cc:654] T d9e82a9592b741e5a90fabd5c4c3481e P c129e37176e24862896138306355891c: Neither blocks nor log segments found. Creating new log.
I20260925 07:29:21.261027 25926 log.cc:824] T d9e82a9592b741e5a90fabd5c4c3481e P 6f62732bcd7f4ff99e9bafb6664615a5: Log is configured to *not* fsync() on all Append() calls
I20260925 07:29:21.261181 25928 tablet_bootstrap.cc:654] T d9e82a9592b741e5a90fabd5c4c3481e P aef05e0706bc4268a182fe5938ba6e51: Neither blocks nor log segments found. Creating new log.
I20260925 07:29:21.261401 25927 log.cc:824] T d9e82a9592b741e5a90fabd5c4c3481e P c129e37176e24862896138306355891c: Log is configured to *not* fsync() on all Append() calls
I20260925 07:29:21.261477 25928 log.cc:824] T d9e82a9592b741e5a90fabd5c4c3481e P aef05e0706bc4268a182fe5938ba6e51: Log is configured to *not* fsync() on all Append() calls
I20260925 07:29:21.261794 25926 tablet_bootstrap.cc:492] T d9e82a9592b741e5a90fabd5c4c3481e P 6f62732bcd7f4ff99e9bafb6664615a5: No bootstrap required, opened a new log
I20260925 07:29:21.261866 25926 ts_tablet_manager.cc:1397] T d9e82a9592b741e5a90fabd5c4c3481e P 6f62732bcd7f4ff99e9bafb6664615a5: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260925 07:29:21.262086 25927 tablet_bootstrap.cc:492] T d9e82a9592b741e5a90fabd5c4c3481e P c129e37176e24862896138306355891c: No bootstrap required, opened a new log
I20260925 07:29:21.262152 25928 tablet_bootstrap.cc:492] T d9e82a9592b741e5a90fabd5c4c3481e P aef05e0706bc4268a182fe5938ba6e51: No bootstrap required, opened a new log
I20260925 07:29:21.262162 25927 ts_tablet_manager.cc:1397] T d9e82a9592b741e5a90fabd5c4c3481e P c129e37176e24862896138306355891c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 07:29:21.262228 25928 ts_tablet_manager.cc:1397] T d9e82a9592b741e5a90fabd5c4c3481e P aef05e0706bc4268a182fe5938ba6e51: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260925 07:29:21.263818 25927 raft_consensus.cc:348] T d9e82a9592b741e5a90fabd5c4c3481e P c129e37176e24862896138306355891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c129e37176e24862896138306355891c" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 40073 } } peers { permanent_uuid: "6f62732bcd7f4ff99e9bafb6664615a5" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 42889 } } peers { permanent_uuid: "aef05e0706bc4268a182fe5938ba6e51" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 34025 } }
I20260925 07:29:21.263969 25927 raft_consensus.cc:374] T d9e82a9592b741e5a90fabd5c4c3481e P c129e37176e24862896138306355891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:29:21.264009 25927 raft_consensus.cc:729] T d9e82a9592b741e5a90fabd5c4c3481e P c129e37176e24862896138306355891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c129e37176e24862896138306355891c, State: Initialized, Role: FOLLOWER
I20260925 07:29:21.264115 25927 consensus_queue.cc:260] T d9e82a9592b741e5a90fabd5c4c3481e P c129e37176e24862896138306355891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c129e37176e24862896138306355891c" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 40073 } } peers { permanent_uuid: "6f62732bcd7f4ff99e9bafb6664615a5" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 42889 } } peers { permanent_uuid: "aef05e0706bc4268a182fe5938ba6e51" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 34025 } }
I20260925 07:29:21.264351 25927 ts_tablet_manager.cc:1428] T d9e82a9592b741e5a90fabd5c4c3481e P c129e37176e24862896138306355891c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 07:29:21.264289 25926 raft_consensus.cc:348] T d9e82a9592b741e5a90fabd5c4c3481e P 6f62732bcd7f4ff99e9bafb6664615a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c129e37176e24862896138306355891c" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 40073 } } peers { permanent_uuid: "6f62732bcd7f4ff99e9bafb6664615a5" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 42889 } } peers { permanent_uuid: "aef05e0706bc4268a182fe5938ba6e51" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 34025 } }
I20260925 07:29:21.264431 25926 raft_consensus.cc:374] T d9e82a9592b741e5a90fabd5c4c3481e P 6f62732bcd7f4ff99e9bafb6664615a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:29:21.264456 25912 heartbeater.cc:503] Master 127.5.158.126:37757 was elected leader, sending a full tablet report...
I20260925 07:29:21.264461 25926 raft_consensus.cc:729] T d9e82a9592b741e5a90fabd5c4c3481e P 6f62732bcd7f4ff99e9bafb6664615a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f62732bcd7f4ff99e9bafb6664615a5, State: Initialized, Role: FOLLOWER
I20260925 07:29:21.264561 25926 consensus_queue.cc:260] T d9e82a9592b741e5a90fabd5c4c3481e P 6f62732bcd7f4ff99e9bafb6664615a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c129e37176e24862896138306355891c" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 40073 } } peers { permanent_uuid: "6f62732bcd7f4ff99e9bafb6664615a5" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 42889 } } peers { permanent_uuid: "aef05e0706bc4268a182fe5938ba6e51" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 34025 } }
I20260925 07:29:21.264516 25928 raft_consensus.cc:348] T d9e82a9592b741e5a90fabd5c4c3481e P aef05e0706bc4268a182fe5938ba6e51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c129e37176e24862896138306355891c" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 40073 } } peers { permanent_uuid: "6f62732bcd7f4ff99e9bafb6664615a5" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 42889 } } peers { permanent_uuid: "aef05e0706bc4268a182fe5938ba6e51" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 34025 } }
I20260925 07:29:21.264631 25928 raft_consensus.cc:374] T d9e82a9592b741e5a90fabd5c4c3481e P aef05e0706bc4268a182fe5938ba6e51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:29:21.264667 25928 raft_consensus.cc:729] T d9e82a9592b741e5a90fabd5c4c3481e P aef05e0706bc4268a182fe5938ba6e51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aef05e0706bc4268a182fe5938ba6e51, State: Initialized, Role: FOLLOWER
I20260925 07:29:21.264779 25926 ts_tablet_manager.cc:1428] T d9e82a9592b741e5a90fabd5c4c3481e P 6f62732bcd7f4ff99e9bafb6664615a5: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260925 07:29:21.264786 25928 consensus_queue.cc:260] T d9e82a9592b741e5a90fabd5c4c3481e P aef05e0706bc4268a182fe5938ba6e51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c129e37176e24862896138306355891c" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 40073 } } peers { permanent_uuid: "6f62732bcd7f4ff99e9bafb6664615a5" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 42889 } } peers { permanent_uuid: "aef05e0706bc4268a182fe5938ba6e51" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 34025 } }
I20260925 07:29:21.264896 25781 heartbeater.cc:503] Master 127.5.158.126:37757 was elected leader, sending a full tablet report...
I20260925 07:29:21.264999 25928 ts_tablet_manager.cc:1428] T d9e82a9592b741e5a90fabd5c4c3481e P aef05e0706bc4268a182fe5938ba6e51: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260925 07:29:21.265117 25650 heartbeater.cc:503] Master 127.5.158.126:37757 was elected leader, sending a full tablet report...
I20260925 07:29:21.344143 25933 raft_consensus.cc:482] T d9e82a9592b741e5a90fabd5c4c3481e P 6f62732bcd7f4ff99e9bafb6664615a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 07:29:21.344311 25933 raft_consensus.cc:504] T d9e82a9592b741e5a90fabd5c4c3481e P 6f62732bcd7f4ff99e9bafb6664615a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c129e37176e24862896138306355891c" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 40073 } } peers { permanent_uuid: "6f62732bcd7f4ff99e9bafb6664615a5" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 42889 } } peers { permanent_uuid: "aef05e0706bc4268a182fe5938ba6e51" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 34025 } }
I20260925 07:29:21.344790 25933 leader_election.cc:288] T d9e82a9592b741e5a90fabd5c4c3481e P 6f62732bcd7f4ff99e9bafb6664615a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c129e37176e24862896138306355891c (127.5.158.67:40073), aef05e0706bc4268a182fe5938ba6e51 (127.5.158.65:34025)
I20260925 07:29:21.347467 25925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:21.345015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:49155 (local address 127.5.158.65:34025)
0925 07:29:21.345066 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:21.345068 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:21.345078 (+ 10us) server_negotiation.cc:407] Connection header received
0925 07:29:21.345178 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:21.345178 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:21.345192 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:21.345259 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:29:21.345432 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:21.345937 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:21.346815 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:21.347163 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:21.347213 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:21.347220 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:21.347361 (+ 141us) server_negotiation.cc:299] Negotiation successful
0925 07:29:21.347408 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 07:29:21.347470 25783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:21.344919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:34025 (local address 127.5.158.66:49155)
0925 07:29:21.344973 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:21.344986 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:21.345032 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:21.345272 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:21.345273 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:21.345274 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:29:21.345354 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:21.345356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:21.346017 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:21.346018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:21.346734 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:21.346735 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:21.347232 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:21.347233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:21.347296 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:21.347299 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 07:29:21.347345 (+ 46us) client_negotiation.cc:240] Negotiation successful
0925 07:29:21.347399 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260925 07:29:21.347528 25922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:21.345043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.158.66:36743 (local address 127.5.158.67:40073)
0925 07:29:21.345114 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:21.345116 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:21.345223 (+ 107us) server_negotiation.cc:407] Connection header received
0925 07:29:21.345276 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:21.345277 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:21.345289 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:21.345336 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 07:29:21.345529 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:21.346065 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:21.346869 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:21.347186 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:21.347269 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:21.347276 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:21.347423 (+ 147us) server_negotiation.cc:299] Negotiation successful
0925 07:29:21.347460 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260925 07:29:21.347779 25935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:21.344926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.67:40073 (local address 127.5.158.66:36743)
0925 07:29:21.345147 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:21.345155 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:21.345212 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:21.345382 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:21.345383 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:21.345386 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 07:29:21.345466 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:21.345468 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:21.346133 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:21.346133 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:21.346790 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:21.346792 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:21.347280 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:21.347281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:21.347348 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:21.347353 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 07:29:21.347404 (+ 51us) client_negotiation.cc:240] Negotiation successful
0925 07:29:21.347437 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"mutex_wait_us":24,"thread_start_us":118,"threads_started":1}
I20260925 07:29:21.347909 25604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9e82a9592b741e5a90fabd5c4c3481e" candidate_uuid: "6f62732bcd7f4ff99e9bafb6664615a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aef05e0706bc4268a182fe5938ba6e51" is_pre_election: true
I20260925 07:29:21.348040 25604 raft_consensus.cc:2454] T d9e82a9592b741e5a90fabd5c4c3481e P aef05e0706bc4268a182fe5938ba6e51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f62732bcd7f4ff99e9bafb6664615a5 in term 0.
I20260925 07:29:21.348090 25866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9e82a9592b741e5a90fabd5c4c3481e" candidate_uuid: "6f62732bcd7f4ff99e9bafb6664615a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c129e37176e24862896138306355891c" is_pre_election: true
I20260925 07:29:21.348240 25866 raft_consensus.cc:2454] T d9e82a9592b741e5a90fabd5c4c3481e P c129e37176e24862896138306355891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f62732bcd7f4ff99e9bafb6664615a5 in term 0.
I20260925 07:29:21.348277 25669 leader_election.cc:302] T d9e82a9592b741e5a90fabd5c4c3481e P 6f62732bcd7f4ff99e9bafb6664615a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f62732bcd7f4ff99e9bafb6664615a5, aef05e0706bc4268a182fe5938ba6e51; no voters:
I20260925 07:29:21.348430 25933 raft_consensus.cc:2781] T d9e82a9592b741e5a90fabd5c4c3481e P 6f62732bcd7f4ff99e9bafb6664615a5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 07:29:21.348474 25933 raft_consensus.cc:482] T d9e82a9592b741e5a90fabd5c4c3481e P 6f62732bcd7f4ff99e9bafb6664615a5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 07:29:21.348502 25933 raft_consensus.cc:3037] T d9e82a9592b741e5a90fabd5c4c3481e P 6f62732bcd7f4ff99e9bafb6664615a5 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:29:21.349243 25933 raft_consensus.cc:504] T d9e82a9592b741e5a90fabd5c4c3481e P 6f62732bcd7f4ff99e9bafb6664615a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c129e37176e24862896138306355891c" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 40073 } } peers { permanent_uuid: "6f62732bcd7f4ff99e9bafb6664615a5" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 42889 } } peers { permanent_uuid: "aef05e0706bc4268a182fe5938ba6e51" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 34025 } }
I20260925 07:29:21.349371 25933 leader_election.cc:288] T d9e82a9592b741e5a90fabd5c4c3481e P 6f62732bcd7f4ff99e9bafb6664615a5 [CANDIDATE]: Term 1 election: Requested vote from peers c129e37176e24862896138306355891c (127.5.158.67:40073), aef05e0706bc4268a182fe5938ba6e51 (127.5.158.65:34025)
I20260925 07:29:21.349510 25604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9e82a9592b741e5a90fabd5c4c3481e" candidate_uuid: "6f62732bcd7f4ff99e9bafb6664615a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aef05e0706bc4268a182fe5938ba6e51"
I20260925 07:29:21.349581 25604 raft_consensus.cc:3037] T d9e82a9592b741e5a90fabd5c4c3481e P aef05e0706bc4268a182fe5938ba6e51 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:29:21.349573 25866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9e82a9592b741e5a90fabd5c4c3481e" candidate_uuid: "6f62732bcd7f4ff99e9bafb6664615a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c129e37176e24862896138306355891c"
I20260925 07:29:21.349676 25866 raft_consensus.cc:3037] T d9e82a9592b741e5a90fabd5c4c3481e P c129e37176e24862896138306355891c [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:29:21.350288 25604 raft_consensus.cc:2454] T d9e82a9592b741e5a90fabd5c4c3481e P aef05e0706bc4268a182fe5938ba6e51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f62732bcd7f4ff99e9bafb6664615a5 in term 1.
I20260925 07:29:21.350482 25669 leader_election.cc:302] T d9e82a9592b741e5a90fabd5c4c3481e P 6f62732bcd7f4ff99e9bafb6664615a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f62732bcd7f4ff99e9bafb6664615a5, aef05e0706bc4268a182fe5938ba6e51; no voters:
I20260925 07:29:21.350551 25866 raft_consensus.cc:2454] T d9e82a9592b741e5a90fabd5c4c3481e P c129e37176e24862896138306355891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f62732bcd7f4ff99e9bafb6664615a5 in term 1.
I20260925 07:29:21.350572 25933 raft_consensus.cc:2781] T d9e82a9592b741e5a90fabd5c4c3481e P 6f62732bcd7f4ff99e9bafb6664615a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:29:21.350695 25933 raft_consensus.cc:686] T d9e82a9592b741e5a90fabd5c4c3481e P 6f62732bcd7f4ff99e9bafb6664615a5 [term 1 LEADER]: Becoming Leader. State: Replica: 6f62732bcd7f4ff99e9bafb6664615a5, State: Running, Role: LEADER
I20260925 07:29:21.350803 25933 consensus_queue.cc:237] T d9e82a9592b741e5a90fabd5c4c3481e P 6f62732bcd7f4ff99e9bafb6664615a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c129e37176e24862896138306355891c" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 40073 } } peers { permanent_uuid: "6f62732bcd7f4ff99e9bafb6664615a5" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 42889 } } peers { permanent_uuid: "aef05e0706bc4268a182fe5938ba6e51" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 34025 } }
W20260925 07:29:21.351572 25782 tablet.cc:2369] T d9e82a9592b741e5a90fabd5c4c3481e P 6f62732bcd7f4ff99e9bafb6664615a5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 07:29:21.351526 25461 catalog_manager.cc:5495] T d9e82a9592b741e5a90fabd5c4c3481e P 6f62732bcd7f4ff99e9bafb6664615a5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f62732bcd7f4ff99e9bafb6664615a5 (127.5.158.66). New cstate: current_term: 1 leader_uuid: "6f62732bcd7f4ff99e9bafb6664615a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c129e37176e24862896138306355891c" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 40073 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f62732bcd7f4ff99e9bafb6664615a5" member_type: VOTER last_known_addr { host: "127.5.158.66" port: 42889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aef05e0706bc4268a182fe5938ba6e51" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 34025 } health_report { overall_health: UNKNOWN } } }
I20260925 07:29:21.371416 25924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:21.369404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41320 (local address 127.5.158.66:42889)
0925 07:29:21.369467 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:21.369469 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:21.369486 (+ 17us) server_negotiation.cc:407] Connection header received
0925 07:29:21.369512 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:21.369513 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:21.369529 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:21.369589 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 07:29:21.369685 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:21.370123 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:21.370785 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:21.370964 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:21.371022 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:21.371135 (+ 113us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 07:29:21.371219 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 07:29:21.371277 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:21.371315 (+ 38us) server_negotiation.cc:299] Negotiation successful
0925 07:29:21.371331 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260925 07:29:21.372943 25604 raft_consensus.cc:1261] T d9e82a9592b741e5a90fabd5c4c3481e P aef05e0706bc4268a182fe5938ba6e51 [term 1 FOLLOWER]: Refusing update from remote peer 6f62732bcd7f4ff99e9bafb6664615a5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:29:21.373026 25866 raft_consensus.cc:1261] T d9e82a9592b741e5a90fabd5c4c3481e P c129e37176e24862896138306355891c [term 1 FOLLOWER]: Refusing update from remote peer 6f62732bcd7f4ff99e9bafb6664615a5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:29:21.373299 25936 consensus_queue.cc:1035] T d9e82a9592b741e5a90fabd5c4c3481e P 6f62732bcd7f4ff99e9bafb6664615a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c129e37176e24862896138306355891c" member_type: VOTER last_known_addr { host: "127.5.158.67" port: 40073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:29:21.373396 25933 consensus_queue.cc:1035] T d9e82a9592b741e5a90fabd5c4c3481e P 6f62732bcd7f4ff99e9bafb6664615a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aef05e0706bc4268a182fe5938ba6e51" member_type: VOTER last_known_addr { host: "127.5.158.65" port: 34025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:29:21.378324 25462 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:57456:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 07:29:21.378408 25462 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:57456:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 07:29:21.379295 25462 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5f0df05593b3442a8ba8f8226e1b06b9: Sending DeleteTablet for 3 replicas of tablet d9e82a9592b741e5a90fabd5c4c3481e
I20260925 07:29:21.379578 25715 tablet_service.cc:1514] Processing DeleteTablet for tablet d9e82a9592b741e5a90fabd5c4c3481e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:29:21 UTC) from {username='slave'} at 127.0.0.1:41316
I20260925 07:29:21.379720 25846 tablet_service.cc:1514] Processing DeleteTablet for tablet d9e82a9592b741e5a90fabd5c4c3481e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:29:21 UTC) from {username='slave'} at 127.0.0.1:56908
I20260925 07:29:21.379827 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 25522
I20260925 07:29:21.379953 25584 tablet_service.cc:1514] Processing DeleteTablet for tablet d9e82a9592b741e5a90fabd5c4c3481e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 07:29:21 UTC) from {username='slave'} at 127.0.0.1:44388
I20260925 07:29:21.381218 25949 tablet_replica.cc:330] T d9e82a9592b741e5a90fabd5c4c3481e P c129e37176e24862896138306355891c: stopping tablet replica
I20260925 07:29:21.381325 25949 raft_consensus.cc:2229] T d9e82a9592b741e5a90fabd5c4c3481e P c129e37176e24862896138306355891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 07:29:21.381428 25949 raft_consensus.cc:2258] T d9e82a9592b741e5a90fabd5c4c3481e P c129e37176e24862896138306355891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:29:21.381805 25949 ts_tablet_manager.cc:1905] T d9e82a9592b741e5a90fabd5c4c3481e P c129e37176e24862896138306355891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 07:29:21.383706 25949 ts_tablet_manager.cc:1918] T d9e82a9592b741e5a90fabd5c4c3481e P c129e37176e24862896138306355891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 07:29:21.383782 25949 log.cc:1196] T d9e82a9592b741e5a90fabd5c4c3481e P c129e37176e24862896138306355891c: Deleting WAL directory at /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790321110323796-5753-0/minicluster-data/ts-2/wal/wals/d9e82a9592b741e5a90fabd5c4c3481e
I20260925 07:29:21.384238 25949 ts_tablet_manager.cc:1939] T d9e82a9592b741e5a90fabd5c4c3481e P c129e37176e24862896138306355891c: Deleting consensus metadata
I20260925 07:29:21.384644 25448 catalog_manager.cc:4841] TS c129e37176e24862896138306355891c (127.5.158.67:40073): tablet d9e82a9592b741e5a90fabd5c4c3481e (table test-table [id=595d124d1b9940e7acdc2f2f88de262a]) successfully deleted
W20260925 07:29:21.386457 25447 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.158.65:34025 (error 108)
I20260925 07:29:21.386765 25948 tablet_replica.cc:330] T d9e82a9592b741e5a90fabd5c4c3481e P 6f62732bcd7f4ff99e9bafb6664615a5: stopping tablet replica
I20260925 07:29:21.386868 25948 raft_consensus.cc:2229] T d9e82a9592b741e5a90fabd5c4c3481e P 6f62732bcd7f4ff99e9bafb6664615a5 [term 1 LEADER]: Raft consensus shutting down.
I20260925 07:29:21.386978 25948 raft_consensus.cc:2258] T d9e82a9592b741e5a90fabd5c4c3481e P 6f62732bcd7f4ff99e9bafb6664615a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 07:29:21.387202 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 25653
I20260925 07:29:21.387768 25920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:21.387291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.65:34025 (local address 127.0.0.1:44398)
0925 07:29:21.387693 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:21.387730 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.65:34025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385}
W20260925 07:29:21.388019 25447 catalog_manager.cc:4584] TS aef05e0706bc4268a182fe5938ba6e51 (127.5.158.65:34025): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d9e82a9592b741e5a90fabd5c4c3481e: Network error: Client connection negotiation failed: client connection to 127.5.158.65:34025: connect: Connection refused (error 111)
I20260925 07:29:21.392452 25920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:21.392328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.66:42889 (local address 127.0.0.1:41328)
0925 07:29:21.392404 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:21.392427 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.66:42889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260925 07:29:21.392446 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 25784
I20260925 07:29:21.396574 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 25431
2026-09-25T07:29:21Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (690 ms)
[----------] 6 tests from EncryptionPolicyTest (8662 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790321110323796-5753-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 07:29:21 dist-test-slave-1drq krb5kdc[25953](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 07:29:21 dist-test-slave-1drq krb5kdc[25953](info): set up 2 sockets
Sep 25 07:29:21 dist-test-slave-1drq krb5kdc[25953](info): commencing operation
krb5kdc: starting...
W20260925 07:29:22.344645 5753 mini_kdc.cc:121] Time spent starting KDC: real 0.927s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Sep 25 07:29:22 dist-test-slave-1drq krb5kdc[25953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321362, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T07:29:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T07:29:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790321110323796-5753-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790321110323796-5753-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 07:29:22.446041 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35879
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790321110323796-5753-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35879
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:29:22.522069 25966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:29:22.522243 25966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:29:22.522264 25966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:29:22.523314 25966 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:29:22.523348 25966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:29:22.523362 25966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:29:22.523375 25966 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:29:22.523386 25966 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:29:22.524708 25966 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790321110323796-5753-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35879
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790321110323796-5753-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35879
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25966
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:29:22.524870 25966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:29:22.525017 25966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:29:22.529525 25974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:22.529536 25971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:22.529659 25966 server_base.cc:1034] running on GCE node
W20260925 07:29:22.529536 25972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:22.530006 25966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:29:22.530285 25966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:29:22.531459 25966 hybrid_clock.cc:648] HybridClock initialized: now 1790321362531451 us; error 27 us; skew 500 ppm
Sep 25 07:29:22 dist-test-slave-1drq krb5kdc[25953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321362, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:29:22.538420 25966 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 07:29:22.538869 25966 webserver.cc:460] Webserver started at http://127.0.0.1:43689/ using document root <none> and password file <none>
I20260925 07:29:22.539095 25966 fs_manager.cc:359] Metadata directory not provided
I20260925 07:29:22.539161 25966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:29:22.539305 25966 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:29:22.540275 25966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790321110323796-5753-0/minicluster-data/master-0/data/instance:
uuid: "5861ee6d9f664ec7a039b0aab36366b2"
format_stamp: "Formatted at 2026-09-25 07:29:22 on dist-test-slave-1drq"
server_key: "6e03204a5f161891c171f35c86852c6f"
server_key_iv: "593ad5a745641818d6281e11c380c8aa"
server_key_version: "encryptionkey@0"
I20260925 07:29:22.540612 25966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790321110323796-5753-0/minicluster-data/master-0/wal/instance:
uuid: "5861ee6d9f664ec7a039b0aab36366b2"
format_stamp: "Formatted at 2026-09-25 07:29:22 on dist-test-slave-1drq"
server_key: "6e03204a5f161891c171f35c86852c6f"
server_key_iv: "593ad5a745641818d6281e11c380c8aa"
server_key_version: "encryptionkey@0"
I20260925 07:29:22.542042 25966 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 07:29:22.542726 25981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:22.542910 25966 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:22.543000 25966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "5861ee6d9f664ec7a039b0aab36366b2"
format_stamp: "Formatted at 2026-09-25 07:29:22 on dist-test-slave-1drq"
server_key: "6e03204a5f161891c171f35c86852c6f"
server_key_iv: "593ad5a745641818d6281e11c380c8aa"
server_key_version: "encryptionkey@0"
I20260925 07:29:22.543072 25966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:29:22.559238 25966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:29:22.561407 25966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:29:22.561560 25966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:29:22.565276 25966 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35879
I20260925 07:29:22.565305 26033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35879 every 8 connection(s)
I20260925 07:29:22.565635 25966 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:29:22.566524 26034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:29:22.568464 26034 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5861ee6d9f664ec7a039b0aab36366b2: Bootstrap starting.
I20260925 07:29:22.568955 26034 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5861ee6d9f664ec7a039b0aab36366b2: Neither blocks nor log segments found. Creating new log.
I20260925 07:29:22.569252 26034 log.cc:824] T 00000000000000000000000000000000 P 5861ee6d9f664ec7a039b0aab36366b2: Log is configured to *not* fsync() on all Append() calls
I20260925 07:29:22.569832 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 25966
I20260925 07:29:22.569962 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790321110323796-5753-0/minicluster-data/master-0/wal/instance
I20260925 07:29:22.570032 26034 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5861ee6d9f664ec7a039b0aab36366b2: No bootstrap required, opened a new log
I20260925 07:29:22.570182 5753 external_mini_cluster.cc:1428] Setting key 44290a60753c32bbeb5bd976acaf0645
I20260925 07:29:22.572235 26034 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5861ee6d9f664ec7a039b0aab36366b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5861ee6d9f664ec7a039b0aab36366b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35879 } }
I20260925 07:29:22.572345 26034 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5861ee6d9f664ec7a039b0aab36366b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:29:22.572364 26034 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5861ee6d9f664ec7a039b0aab36366b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5861ee6d9f664ec7a039b0aab36366b2, State: Initialized, Role: FOLLOWER
I20260925 07:29:22.572470 26034 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5861ee6d9f664ec7a039b0aab36366b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5861ee6d9f664ec7a039b0aab36366b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35879 } }
I20260925 07:29:22.572537 26034 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5861ee6d9f664ec7a039b0aab36366b2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 07:29:22.572557 26034 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5861ee6d9f664ec7a039b0aab36366b2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 07:29:22.572611 26034 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5861ee6d9f664ec7a039b0aab36366b2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:29:22.573122 26034 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5861ee6d9f664ec7a039b0aab36366b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5861ee6d9f664ec7a039b0aab36366b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35879 } }
I20260925 07:29:22.573208 26034 leader_election.cc:302] T 00000000000000000000000000000000 P 5861ee6d9f664ec7a039b0aab36366b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5861ee6d9f664ec7a039b0aab36366b2; no voters:
I20260925 07:29:22.573421 26034 leader_election.cc:288] T 00000000000000000000000000000000 P 5861ee6d9f664ec7a039b0aab36366b2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 07:29:22.573437 26039 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5861ee6d9f664ec7a039b0aab36366b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:29:22.573550 26039 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5861ee6d9f664ec7a039b0aab36366b2 [term 1 LEADER]: Becoming Leader. State: Replica: 5861ee6d9f664ec7a039b0aab36366b2, State: Running, Role: LEADER
I20260925 07:29:22.573623 26039 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5861ee6d9f664ec7a039b0aab36366b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5861ee6d9f664ec7a039b0aab36366b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35879 } }
I20260925 07:29:22.573669 26034 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5861ee6d9f664ec7a039b0aab36366b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:29:22.573899 26039 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5861ee6d9f664ec7a039b0aab36366b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5861ee6d9f664ec7a039b0aab36366b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5861ee6d9f664ec7a039b0aab36366b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35879 } } }
I20260925 07:29:22.573956 26039 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5861ee6d9f664ec7a039b0aab36366b2 [sys.catalog]: This master's current role is: LEADER
I20260925 07:29:22.574069 26039 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5861ee6d9f664ec7a039b0aab36366b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5861ee6d9f664ec7a039b0aab36366b2. Latest consensus state: current_term: 1 leader_uuid: "5861ee6d9f664ec7a039b0aab36366b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5861ee6d9f664ec7a039b0aab36366b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35879 } } }
I20260925 07:29:22.574121 26039 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5861ee6d9f664ec7a039b0aab36366b2 [sys.catalog]: This master's current role is: LEADER
W20260925 07:29:22.575297 26055 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5861ee6d9f664ec7a039b0aab36366b2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 07:29:22.575353 26055 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 07:29:22.575692 26043 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 07:29:22.575911 26043 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 25 07:29:22 dist-test-slave-1drq krb5kdc[25953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321362, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260925 07:29:22.577415 26043 catalog_manager.cc:1347] Generated new cluster ID: 95889f1b6b5a4a91bbb4d2b71fbdd5c4
I20260925 07:29:22.577463 26043 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 07:29:22.581053 26038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:22.570702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42479 (local address 127.0.0.1:35879)
0925 07:29:22.570869 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:22.570882 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:22.570910 (+ 28us) server_negotiation.cc:407] Connection header received
0925 07:29:22.571299 (+ 389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:22.571305 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:22.571346 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:22.571450 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:22.572143 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:22.573473 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:22.574491 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:22.574966 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:22.575073 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:22.577757 (+ 2684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:22.577760 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:22.577761 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:29:22.577770 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:22.579533 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:22.579925 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:22.579928 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:22.579928 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:22.580015 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:22.580247 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:22.580248 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:22.580249 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:22.580479 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:22.580687 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:22.580781 (+ 94us) server_negotiation.cc:299] Negotiation successful
0925 07:29:22.580874 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":90,"threads_started":1}
I20260925 07:29:22.582183 5753 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 07:29:22.591176 26038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:22.583529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36399 (local address 127.0.0.1:35879)
0925 07:29:22.583563 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:22.583565 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:22.585242 (+ 1677us) server_negotiation.cc:407] Connection header received
0925 07:29:22.585317 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:22.585317 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:22.585325 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:22.585394 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:22.586074 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:22.586559 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:22.587223 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:22.587420 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:22.587485 (+ 65us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:22.588576 (+ 1091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:22.588577 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:22.588578 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:29:22.588584 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:22.589985 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:22.590357 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:22.590358 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:22.590359 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:22.590410 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:22.590644 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:22.590645 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:22.590645 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:22.590746 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:22.590848 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:22.591020 (+ 172us) server_negotiation.cc:299] Negotiation successful
0925 07:29:22.591079 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 07:29:22.604981 26043 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 07:29:22.605620 26043 catalog_manager.cc:1504] Loading token signing keys...
I20260925 07:29:22.612833 26043 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5861ee6d9f664ec7a039b0aab36366b2: Generated new TSK 0
I20260925 07:29:22.612982 26043 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 07:29:22.618839 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 25966
W20260925 07:29:22.623201 25951 connection.cc:535] client connection to 127.0.0.1:35879 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35879): Connection reset by peer (error 104)
2026-09-25T07:29:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1219 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790321110323796-5753-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 07:29:22 dist-test-slave-1drq krb5kdc[26065](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 07:29:22 dist-test-slave-1drq krb5kdc[26065](info): set up 2 sockets
Sep 25 07:29:22 dist-test-slave-1drq krb5kdc[26065](info): commencing operation
krb5kdc: starting...
W20260925 07:29:23.568861 5753 mini_kdc.cc:121] Time spent starting KDC: real 0.933s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Sep 25 07:29:23 dist-test-slave-1drq krb5kdc[26065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321363, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T07:29:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T07:29:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790321110323796-5753-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790321110323796-5753-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 07:29:23.671095 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34195
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790321110323796-5753-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34195
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:29:23.740945 26078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:29:23.741196 26078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:29:23.741231 26078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:29:23.742184 26078 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:29:23.742241 26078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:29:23.742262 26078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:29:23.742285 26078 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:29:23.742312 26078 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:29:23.743618 26078 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790321110323796-5753-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34195
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790321110323796-5753-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34195
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26078
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:29:23.743846 26078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:29:23.744092 26078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:29:23.748270 26086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:23.748275 26083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:23.748409 26078 server_base.cc:1034] running on GCE node
W20260925 07:29:23.748283 26084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:23.748713 26078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:29:23.748947 26078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:29:23.750078 26078 hybrid_clock.cc:648] HybridClock initialized: now 1790321363750064 us; error 29 us; skew 500 ppm
Sep 25 07:29:23 dist-test-slave-1drq krb5kdc[26065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321363, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:29:23.752300 26078 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 07:29:23.752728 26078 webserver.cc:460] Webserver started at http://127.0.0.1:40421/ using document root <none> and password file <none>
I20260925 07:29:23.752946 26078 fs_manager.cc:359] Metadata directory not provided
I20260925 07:29:23.753017 26078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:29:23.753180 26078 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:29:23.754114 26078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790321110323796-5753-0/minicluster-data/master-0/data/instance:
uuid: "f4b347bcb25d4c49928bbf62ebea8496"
format_stamp: "Formatted at 2026-09-25 07:29:23 on dist-test-slave-1drq"
server_key: "3d851e46eef9702d2b16f366b500732c"
server_key_iv: "9666ff18039dab2574aad73d231cdd15"
server_key_version: "encryptionkey@0"
I20260925 07:29:23.754451 26078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790321110323796-5753-0/minicluster-data/master-0/wal/instance:
uuid: "f4b347bcb25d4c49928bbf62ebea8496"
format_stamp: "Formatted at 2026-09-25 07:29:23 on dist-test-slave-1drq"
server_key: "3d851e46eef9702d2b16f366b500732c"
server_key_iv: "9666ff18039dab2574aad73d231cdd15"
server_key_version: "encryptionkey@0"
I20260925 07:29:23.755595 26078 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 07:29:23.756294 26093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:23.756455 26078 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:29:23.756548 26078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "f4b347bcb25d4c49928bbf62ebea8496"
format_stamp: "Formatted at 2026-09-25 07:29:23 on dist-test-slave-1drq"
server_key: "3d851e46eef9702d2b16f366b500732c"
server_key_iv: "9666ff18039dab2574aad73d231cdd15"
server_key_version: "encryptionkey@0"
I20260925 07:29:23.756641 26078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:29:23.768955 26078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:29:23.770649 26078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:29:23.770802 26078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:29:23.775316 26078 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34195
I20260925 07:29:23.775357 26145 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34195 every 8 connection(s)
I20260925 07:29:23.775744 26078 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:29:23.778044 26146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:29:23.780900 26146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4b347bcb25d4c49928bbf62ebea8496: Bootstrap starting.
I20260925 07:29:23.781504 26146 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f4b347bcb25d4c49928bbf62ebea8496: Neither blocks nor log segments found. Creating new log.
I20260925 07:29:23.781771 26146 log.cc:824] T 00000000000000000000000000000000 P f4b347bcb25d4c49928bbf62ebea8496: Log is configured to *not* fsync() on all Append() calls
I20260925 07:29:23.782367 26146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4b347bcb25d4c49928bbf62ebea8496: No bootstrap required, opened a new log
I20260925 07:29:23.783721 26146 raft_consensus.cc:348] T 00000000000000000000000000000000 P f4b347bcb25d4c49928bbf62ebea8496 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4b347bcb25d4c49928bbf62ebea8496" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34195 } }
I20260925 07:29:23.783864 26146 raft_consensus.cc:374] T 00000000000000000000000000000000 P f4b347bcb25d4c49928bbf62ebea8496 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:29:23.783905 26146 raft_consensus.cc:729] T 00000000000000000000000000000000 P f4b347bcb25d4c49928bbf62ebea8496 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4b347bcb25d4c49928bbf62ebea8496, State: Initialized, Role: FOLLOWER
I20260925 07:29:23.784065 26146 consensus_queue.cc:260] T 00000000000000000000000000000000 P f4b347bcb25d4c49928bbf62ebea8496 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4b347bcb25d4c49928bbf62ebea8496" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34195 } }
I20260925 07:29:23.784154 26146 raft_consensus.cc:388] T 00000000000000000000000000000000 P f4b347bcb25d4c49928bbf62ebea8496 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 07:29:23.784191 26146 raft_consensus.cc:482] T 00000000000000000000000000000000 P f4b347bcb25d4c49928bbf62ebea8496 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 07:29:23.784236 26146 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f4b347bcb25d4c49928bbf62ebea8496 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:29:23.784673 26146 raft_consensus.cc:504] T 00000000000000000000000000000000 P f4b347bcb25d4c49928bbf62ebea8496 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4b347bcb25d4c49928bbf62ebea8496" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34195 } }
I20260925 07:29:23.784785 26146 leader_election.cc:302] T 00000000000000000000000000000000 P f4b347bcb25d4c49928bbf62ebea8496 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f4b347bcb25d4c49928bbf62ebea8496; no voters:
I20260925 07:29:23.784960 26146 leader_election.cc:288] T 00000000000000000000000000000000 P f4b347bcb25d4c49928bbf62ebea8496 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 07:29:23.785010 26149 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f4b347bcb25d4c49928bbf62ebea8496 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:29:23.785148 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 26078
I20260925 07:29:23.785183 26149 raft_consensus.cc:686] T 00000000000000000000000000000000 P f4b347bcb25d4c49928bbf62ebea8496 [term 1 LEADER]: Becoming Leader. State: Replica: f4b347bcb25d4c49928bbf62ebea8496, State: Running, Role: LEADER
I20260925 07:29:23.785205 26146 sys_catalog.cc:564] T 00000000000000000000000000000000 P f4b347bcb25d4c49928bbf62ebea8496 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:29:23.785285 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790321110323796-5753-0/minicluster-data/master-0/wal/instance
I20260925 07:29:23.785284 26149 consensus_queue.cc:237] T 00000000000000000000000000000000 P f4b347bcb25d4c49928bbf62ebea8496 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4b347bcb25d4c49928bbf62ebea8496" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34195 } }
I20260925 07:29:23.785516 5753 external_mini_cluster.cc:1428] Setting key 17af346cc4d35a07013cd94c9f2a5906
I20260925 07:29:23.785602 26149 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4b347bcb25d4c49928bbf62ebea8496 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f4b347bcb25d4c49928bbf62ebea8496. Latest consensus state: current_term: 1 leader_uuid: "f4b347bcb25d4c49928bbf62ebea8496" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4b347bcb25d4c49928bbf62ebea8496" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34195 } } }
I20260925 07:29:23.785665 26149 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4b347bcb25d4c49928bbf62ebea8496 [sys.catalog]: This master's current role is: LEADER
I20260925 07:29:23.785758 26150 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4b347bcb25d4c49928bbf62ebea8496 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f4b347bcb25d4c49928bbf62ebea8496" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4b347bcb25d4c49928bbf62ebea8496" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34195 } } }
I20260925 07:29:23.785847 26150 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4b347bcb25d4c49928bbf62ebea8496 [sys.catalog]: This master's current role is: LEADER
W20260925 07:29:23.786899 26166 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f4b347bcb25d4c49928bbf62ebea8496: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 07:29:23.786988 26166 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 07:29:23.787082 26156 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 07:29:23.787269 26156 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 07:29:23.789064 26156 catalog_manager.cc:1347] Generated new cluster ID: 4ca6567263314b09aabf28194567b971
I20260925 07:29:23.789186 26156 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 07:29:23.790968 26163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:23.786259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37031 (local address 127.0.0.1:34195)
0925 07:29:23.786434 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:23.786444 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:23.786473 (+ 29us) server_negotiation.cc:407] Connection header received
0925 07:29:23.786583 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:23.786590 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:23.786658 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:23.787026 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:23.787132 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:23.788002 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:23.789102 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:23.789569 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:23.789679 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:23.790362 (+ 683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:23.790366 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:23.790366 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:29:23.790532 (+ 166us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:23.790560 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:29:23.790572 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:23.790664 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:23.790741 (+ 77us) server_negotiation.cc:299] Negotiation successful
0925 07:29:23.790769 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":96,"threads_started":1}
I20260925 07:29:23.791940 5753 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 07:29:23.795994 26163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:23.793295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38801 (local address 127.0.0.1:34195)
0925 07:29:23.793331 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:23.793333 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:23.793410 (+ 77us) server_negotiation.cc:407] Connection header received
0925 07:29:23.793454 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:23.793455 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:23.793464 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:23.793506 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:23.793654 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:23.794100 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:23.794814 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:23.795044 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:23.795136 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:23.795769 (+ 633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:23.795770 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:23.795770 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:29:23.795775 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:23.795787 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:29:23.795790 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:23.795841 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:23.795898 (+ 57us) server_negotiation.cc:299] Negotiation successful
0925 07:29:23.795908 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 07:29:23.822731 26156 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 07:29:23.823195 26156 catalog_manager.cc:1504] Loading token signing keys...
I20260925 07:29:23.825811 26156 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f4b347bcb25d4c49928bbf62ebea8496: Generated new TSK 0
I20260925 07:29:23.825963 26156 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 07:29:23.832347 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 26078
W20260925 07:29:23.836385 26063 connection.cc:535] client connection to 127.0.0.1:34195 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34195): Connection reset by peer (error 104)
2026-09-25T07:29:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1213 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790321110323796-5753-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 07:29:23 dist-test-slave-1drq krb5kdc[26177](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 07:29:23 dist-test-slave-1drq krb5kdc[26177](info): set up 2 sockets
Sep 25 07:29:23 dist-test-slave-1drq krb5kdc[26177](info): commencing operation
krb5kdc: starting...
W20260925 07:29:24.780967 5753 mini_kdc.cc:121] Time spent starting KDC: real 0.931s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Sep 25 07:29:24 dist-test-slave-1drq krb5kdc[26177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321364, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T07:29:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T07:29:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790321110323796-5753-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790321110323796-5753-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 07:29:24.885557 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46659
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790321110323796-5753-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46659
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:29:24.956467 26190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:29:24.956661 26190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:29:24.956696 26190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:29:24.957744 26190 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:29:24.957803 26190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:29:24.957826 26190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:29:24.957851 26190 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:29:24.957880 26190 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:29:24.959306 26190 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790321110323796-5753-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46659
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790321110323796-5753-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46659
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26190
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:29:24.959551 26190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:29:24.959765 26190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:29:24.964231 26198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:24.964267 26195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:24.964346 26190 server_base.cc:1034] running on GCE node
W20260925 07:29:24.964287 26196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:24.964665 26190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:29:24.964874 26190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:29:24.966048 26190 hybrid_clock.cc:648] HybridClock initialized: now 1790321364966037 us; error 39 us; skew 500 ppm
Sep 25 07:29:24 dist-test-slave-1drq krb5kdc[26177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321364, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:29:24.968429 26190 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 07:29:24.968851 26190 webserver.cc:460] Webserver started at http://127.0.0.1:39791/ using document root <none> and password file <none>
I20260925 07:29:24.969059 26190 fs_manager.cc:359] Metadata directory not provided
I20260925 07:29:24.969154 26190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:29:24.969290 26190 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:29:24.970222 26190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790321110323796-5753-0/minicluster-data/master-0/data/instance:
uuid: "38d0ce1ca313449ca32124ea5ad69e91"
format_stamp: "Formatted at 2026-09-25 07:29:24 on dist-test-slave-1drq"
server_key: "95a2145330321833661c21d6efcbd71d"
server_key_iv: "510ef27b5d97b2cc4217958e35715992"
server_key_version: "encryptionkey@0"
I20260925 07:29:24.970558 26190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790321110323796-5753-0/minicluster-data/master-0/wal/instance:
uuid: "38d0ce1ca313449ca32124ea5ad69e91"
format_stamp: "Formatted at 2026-09-25 07:29:24 on dist-test-slave-1drq"
server_key: "95a2145330321833661c21d6efcbd71d"
server_key_iv: "510ef27b5d97b2cc4217958e35715992"
server_key_version: "encryptionkey@0"
I20260925 07:29:24.971750 26190 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 07:29:24.972414 26205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:24.972575 26190 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:29:24.972664 26190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "38d0ce1ca313449ca32124ea5ad69e91"
format_stamp: "Formatted at 2026-09-25 07:29:24 on dist-test-slave-1drq"
server_key: "95a2145330321833661c21d6efcbd71d"
server_key_iv: "510ef27b5d97b2cc4217958e35715992"
server_key_version: "encryptionkey@0"
I20260925 07:29:24.972756 26190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:29:24.975646 26190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:29:24.977578 26190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:29:24.977818 26190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:29:24.981537 26190 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46659
I20260925 07:29:24.981580 26257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46659 every 8 connection(s)
I20260925 07:29:24.981881 26190 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:29:24.982669 26258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:29:24.984469 26258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38d0ce1ca313449ca32124ea5ad69e91: Bootstrap starting.
I20260925 07:29:24.984913 26258 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 38d0ce1ca313449ca32124ea5ad69e91: Neither blocks nor log segments found. Creating new log.
I20260925 07:29:24.985198 26258 log.cc:824] T 00000000000000000000000000000000 P 38d0ce1ca313449ca32124ea5ad69e91: Log is configured to *not* fsync() on all Append() calls
I20260925 07:29:24.985750 26258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38d0ce1ca313449ca32124ea5ad69e91: No bootstrap required, opened a new log
I20260925 07:29:24.987089 26258 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38d0ce1ca313449ca32124ea5ad69e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38d0ce1ca313449ca32124ea5ad69e91" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46659 } }
I20260925 07:29:24.987195 26258 raft_consensus.cc:374] T 00000000000000000000000000000000 P 38d0ce1ca313449ca32124ea5ad69e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:29:24.987214 26258 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38d0ce1ca313449ca32124ea5ad69e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38d0ce1ca313449ca32124ea5ad69e91, State: Initialized, Role: FOLLOWER
I20260925 07:29:24.987332 26258 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38d0ce1ca313449ca32124ea5ad69e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38d0ce1ca313449ca32124ea5ad69e91" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46659 } }
I20260925 07:29:24.987406 26258 raft_consensus.cc:388] T 00000000000000000000000000000000 P 38d0ce1ca313449ca32124ea5ad69e91 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 07:29:24.987426 26258 raft_consensus.cc:482] T 00000000000000000000000000000000 P 38d0ce1ca313449ca32124ea5ad69e91 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 07:29:24.987450 26258 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38d0ce1ca313449ca32124ea5ad69e91 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:29:24.987850 26258 raft_consensus.cc:504] T 00000000000000000000000000000000 P 38d0ce1ca313449ca32124ea5ad69e91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38d0ce1ca313449ca32124ea5ad69e91" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46659 } }
I20260925 07:29:24.987920 26258 leader_election.cc:302] T 00000000000000000000000000000000 P 38d0ce1ca313449ca32124ea5ad69e91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 38d0ce1ca313449ca32124ea5ad69e91; no voters:
I20260925 07:29:24.988022 26258 leader_election.cc:288] T 00000000000000000000000000000000 P 38d0ce1ca313449ca32124ea5ad69e91 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 07:29:24.988082 26261 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 38d0ce1ca313449ca32124ea5ad69e91 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:29:24.988250 26258 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38d0ce1ca313449ca32124ea5ad69e91 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:29:24.988263 26261 raft_consensus.cc:686] T 00000000000000000000000000000000 P 38d0ce1ca313449ca32124ea5ad69e91 [term 1 LEADER]: Becoming Leader. State: Replica: 38d0ce1ca313449ca32124ea5ad69e91, State: Running, Role: LEADER
I20260925 07:29:24.988407 26261 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38d0ce1ca313449ca32124ea5ad69e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38d0ce1ca313449ca32124ea5ad69e91" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46659 } }
I20260925 07:29:24.988960 26262 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38d0ce1ca313449ca32124ea5ad69e91 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "38d0ce1ca313449ca32124ea5ad69e91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38d0ce1ca313449ca32124ea5ad69e91" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46659 } } }
I20260925 07:29:24.988971 26263 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38d0ce1ca313449ca32124ea5ad69e91 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38d0ce1ca313449ca32124ea5ad69e91. Latest consensus state: current_term: 1 leader_uuid: "38d0ce1ca313449ca32124ea5ad69e91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38d0ce1ca313449ca32124ea5ad69e91" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46659 } } }
I20260925 07:29:24.989068 26263 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38d0ce1ca313449ca32124ea5ad69e91 [sys.catalog]: This master's current role is: LEADER
I20260925 07:29:24.989048 26262 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38d0ce1ca313449ca32124ea5ad69e91 [sys.catalog]: This master's current role is: LEADER
I20260925 07:29:24.989207 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 26190
I20260925 07:29:24.989324 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790321110323796-5753-0/minicluster-data/master-0/wal/instance
I20260925 07:29:24.989475 26270 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 07:29:24.989526 5753 external_mini_cluster.cc:1428] Setting key bf883e791a1832194c360bfcc5e1fd37
I20260925 07:29:24.990034 26270 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 07:29:24.991554 26278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:24.989974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49239 (local address 127.0.0.1:46659)
0925 07:29:24.990143 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:24.990153 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:24.990184 (+ 31us) server_negotiation.cc:407] Connection header received
0925 07:29:24.990305 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:24.990312 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:24.990344 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:24.990489 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:24.991073 (+ 584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:24.991090 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:24.991090 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:29:24.991094 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:24.991149 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:29:24.991161 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:24.991221 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:24.991374 (+ 153us) server_negotiation.cc:299] Negotiation successful
0925 07:29:24.991409 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":72,"threads_started":1}
I20260925 07:29:24.992455 5753 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 07:29:24.993782 26270 catalog_manager.cc:1347] Generated new cluster ID: c14105a6bb6245e4ae61ac1b9bd7fb19
I20260925 07:29:24.993878 26270 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 07:29:24.998894 26278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:24.993870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55705 (local address 127.0.0.1:46659)
0925 07:29:24.997150 (+ 3280us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:24.997152 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:24.998067 (+ 915us) server_negotiation.cc:407] Connection header received
0925 07:29:24.998139 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:24.998140 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:24.998145 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:24.998210 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:24.998705 (+ 495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:24.998706 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:24.998706 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:29:24.998710 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:24.998721 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:29:24.998725 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:24.998776 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:24.998827 (+ 51us) server_negotiation.cc:299] Negotiation successful
0925 07:29:24.998840 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3263}
I20260925 07:29:25.000340 26270 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 07:29:25.000478 26270 catalog_manager.cc:1504] Loading token signing keys...
I20260925 07:29:25.009677 26270 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 38d0ce1ca313449ca32124ea5ad69e91: Generated new TSK 0
I20260925 07:29:25.009842 26270 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 07:29:25.015998 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 26190
2026-09-25T07:29:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1183 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790321110323796-5753-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 07:29:25 dist-test-slave-1drq krb5kdc[26289](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 07:29:25 dist-test-slave-1drq krb5kdc[26289](info): set up 2 sockets
Sep 25 07:29:25 dist-test-slave-1drq krb5kdc[26289](info): commencing operation
krb5kdc: starting...
W20260925 07:29:25.972784 5753 mini_kdc.cc:121] Time spent starting KDC: real 0.940s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Sep 25 07:29:26 dist-test-slave-1drq krb5kdc[26289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T07:29:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T07:29:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790321110323796-5753-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790321110323796-5753-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 07:29:26.073474 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33901
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790321110323796-5753-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33901
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:29:26.143487 26302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:29:26.143639 26302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:29:26.143658 26302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:29:26.144541 26302 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:29:26.144577 26302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:29:26.144589 26302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:29:26.144599 26302 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:29:26.144609 26302 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:29:26.145936 26302 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790321110323796-5753-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33901
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790321110323796-5753-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33901
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26302
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:29:26.146085 26302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:29:26.146210 26302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:29:26.150336 26310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:26.150377 26302 server_base.cc:1034] running on GCE node
W20260925 07:29:26.150341 26308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:26.150326 26307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:26.150804 26302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:29:26.151034 26302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:29:26.152217 26302 hybrid_clock.cc:648] HybridClock initialized: now 1790321366152199 us; error 35 us; skew 500 ppm
Sep 25 07:29:26 dist-test-slave-1drq krb5kdc[26289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321366, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:29:26.154294 26302 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 07:29:26.154600 26302 webserver.cc:460] Webserver started at http://127.0.0.1:33393/ using document root <none> and password file <none>
I20260925 07:29:26.154759 26302 fs_manager.cc:359] Metadata directory not provided
I20260925 07:29:26.154810 26302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:29:26.154902 26302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:29:26.155740 26302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790321110323796-5753-0/minicluster-data/master-0/data/instance:
uuid: "a4260056f556441383a45f1e3744e75b"
format_stamp: "Formatted at 2026-09-25 07:29:26 on dist-test-slave-1drq"
server_key: "24ffad1788c89a63ce26ed1c670cf584"
server_key_iv: "c3957d3a09f350c9b2c9e85ff62d2d0f"
server_key_version: "encryptionkey@0"
I20260925 07:29:26.156011 26302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790321110323796-5753-0/minicluster-data/master-0/wal/instance:
uuid: "a4260056f556441383a45f1e3744e75b"
format_stamp: "Formatted at 2026-09-25 07:29:26 on dist-test-slave-1drq"
server_key: "24ffad1788c89a63ce26ed1c670cf584"
server_key_iv: "c3957d3a09f350c9b2c9e85ff62d2d0f"
server_key_version: "encryptionkey@0"
I20260925 07:29:26.157114 26302 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 07:29:26.157728 26317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:26.157908 26302 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:29:26.157984 26302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "a4260056f556441383a45f1e3744e75b"
format_stamp: "Formatted at 2026-09-25 07:29:26 on dist-test-slave-1drq"
server_key: "24ffad1788c89a63ce26ed1c670cf584"
server_key_iv: "c3957d3a09f350c9b2c9e85ff62d2d0f"
server_key_version: "encryptionkey@0"
I20260925 07:29:26.158032 26302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:29:26.174484 26302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:29:26.176329 26302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:29:26.176436 26302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:29:26.180065 26302 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33901
I20260925 07:29:26.180123 26369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33901 every 8 connection(s)
I20260925 07:29:26.180470 26302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:29:26.181438 26370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:29:26.183207 26370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4260056f556441383a45f1e3744e75b: Bootstrap starting.
I20260925 07:29:26.183676 26370 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a4260056f556441383a45f1e3744e75b: Neither blocks nor log segments found. Creating new log.
I20260925 07:29:26.183918 26370 log.cc:824] T 00000000000000000000000000000000 P a4260056f556441383a45f1e3744e75b: Log is configured to *not* fsync() on all Append() calls
I20260925 07:29:26.184494 26370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4260056f556441383a45f1e3744e75b: No bootstrap required, opened a new log
I20260925 07:29:26.185788 26370 raft_consensus.cc:348] T 00000000000000000000000000000000 P a4260056f556441383a45f1e3744e75b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4260056f556441383a45f1e3744e75b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33901 } }
I20260925 07:29:26.185925 26370 raft_consensus.cc:374] T 00000000000000000000000000000000 P a4260056f556441383a45f1e3744e75b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:29:26.185964 26370 raft_consensus.cc:729] T 00000000000000000000000000000000 P a4260056f556441383a45f1e3744e75b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4260056f556441383a45f1e3744e75b, State: Initialized, Role: FOLLOWER
I20260925 07:29:26.186100 26370 consensus_queue.cc:260] T 00000000000000000000000000000000 P a4260056f556441383a45f1e3744e75b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4260056f556441383a45f1e3744e75b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33901 } }
I20260925 07:29:26.186182 26370 raft_consensus.cc:388] T 00000000000000000000000000000000 P a4260056f556441383a45f1e3744e75b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 07:29:26.186220 26370 raft_consensus.cc:482] T 00000000000000000000000000000000 P a4260056f556441383a45f1e3744e75b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 07:29:26.186264 26370 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a4260056f556441383a45f1e3744e75b [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:29:26.186676 26370 raft_consensus.cc:504] T 00000000000000000000000000000000 P a4260056f556441383a45f1e3744e75b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4260056f556441383a45f1e3744e75b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33901 } }
I20260925 07:29:26.186779 26370 leader_election.cc:302] T 00000000000000000000000000000000 P a4260056f556441383a45f1e3744e75b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a4260056f556441383a45f1e3744e75b; no voters:
I20260925 07:29:26.186939 26370 leader_election.cc:288] T 00000000000000000000000000000000 P a4260056f556441383a45f1e3744e75b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 07:29:26.186964 26373 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a4260056f556441383a45f1e3744e75b [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:29:26.186998 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 26302
I20260925 07:29:26.187063 26373 raft_consensus.cc:686] T 00000000000000000000000000000000 P a4260056f556441383a45f1e3744e75b [term 1 LEADER]: Becoming Leader. State: Replica: a4260056f556441383a45f1e3744e75b, State: Running, Role: LEADER
I20260925 07:29:26.187096 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790321110323796-5753-0/minicluster-data/master-0/wal/instance
I20260925 07:29:26.187121 26373 consensus_queue.cc:237] T 00000000000000000000000000000000 P a4260056f556441383a45f1e3744e75b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4260056f556441383a45f1e3744e75b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33901 } }
I20260925 07:29:26.187173 26370 sys_catalog.cc:564] T 00000000000000000000000000000000 P a4260056f556441383a45f1e3744e75b [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:29:26.187274 5753 external_mini_cluster.cc:1428] Setting key 0ed5873da2e2b049e40cc7364d26dfae
I20260925 07:29:26.187418 26373 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4260056f556441383a45f1e3744e75b [sys.catalog]: SysCatalogTable state changed. Reason: New leader a4260056f556441383a45f1e3744e75b. Latest consensus state: current_term: 1 leader_uuid: "a4260056f556441383a45f1e3744e75b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4260056f556441383a45f1e3744e75b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33901 } } }
I20260925 07:29:26.187551 26373 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4260056f556441383a45f1e3744e75b [sys.catalog]: This master's current role is: LEADER
I20260925 07:29:26.187798 26382 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 07:29:26.187928 26374 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4260056f556441383a45f1e3744e75b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a4260056f556441383a45f1e3744e75b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4260056f556441383a45f1e3744e75b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33901 } } }
I20260925 07:29:26.188052 26374 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4260056f556441383a45f1e3744e75b [sys.catalog]: This master's current role is: LEADER
I20260925 07:29:26.188264 26382 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 07:29:26.189842 26382 catalog_manager.cc:1347] Generated new cluster ID: 9deefec7aa7c42f8be10847c63b53238
I20260925 07:29:26.189890 26382 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 07:29:26 dist-test-slave-1drq krb5kdc[26289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260925 07:29:26.196501 26384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:26.187801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36431 (local address 127.0.0.1:33901)
0925 07:29:26.188134 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:26.188142 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:26.188376 (+ 234us) server_negotiation.cc:407] Connection header received
0925 07:29:26.188490 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:26.188497 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:26.188547 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:26.188657 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:26.189419 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:26.190414 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:26.191166 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:26.191607 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:26.191726 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:26.193775 (+ 2049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:26.193778 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:26.193779 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:29:26.193788 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:26.195144 (+ 1356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:26.195474 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:26.195476 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:26.195477 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:26.195590 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:26.195786 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:26.195787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:26.195787 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:26.195984 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:26.196130 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:26.196207 (+ 77us) server_negotiation.cc:299] Negotiation successful
0925 07:29:26.196322 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":66,"threads_started":1}
I20260925 07:29:26.197230 5753 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 07:29:26.197957 26382 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 07:29:26.198516 26382 catalog_manager.cc:1504] Loading token signing keys...
I20260925 07:29:26.203820 26384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:26.198573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38267 (local address 127.0.0.1:33901)
0925 07:29:26.198609 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:26.198610 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:26.198719 (+ 109us) server_negotiation.cc:407] Connection header received
0925 07:29:26.198784 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:26.198785 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:26.198792 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:26.198829 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:26.199255 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:26.199673 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:26.200402 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:26.200608 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:26.200688 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:26.201743 (+ 1055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:26.201744 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:26.201744 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:29:26.201750 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:26.202865 (+ 1115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:26.203167 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:26.203168 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:26.203168 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:26.203211 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:26.203409 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:26.203409 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:26.203410 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:26.203497 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:26.203580 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:26.203674 (+ 94us) server_negotiation.cc:299] Negotiation successful
0925 07:29:26.203742 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 07:29:26.204576 26382 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a4260056f556441383a45f1e3744e75b: Generated new TSK 0
I20260925 07:29:26.204720 26382 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 07:29:26.206776 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 26302
2026-09-25T07:29:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1191 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790321110323796-5753-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 07:29:26 dist-test-slave-1drq krb5kdc[26401](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 07:29:26 dist-test-slave-1drq krb5kdc[26401](info): set up 2 sockets
Sep 25 07:29:26 dist-test-slave-1drq krb5kdc[26401](info): commencing operation
krb5kdc: starting...
W20260925 07:29:27.148684 5753 mini_kdc.cc:121] Time spent starting KDC: real 0.924s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Sep 25 07:29:27 dist-test-slave-1drq krb5kdc[26401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T07:29:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T07:29:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790321110323796-5753-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790321110323796-5753-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 07:29:27.247758 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39293
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790321110323796-5753-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39293
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:29:27.318442 26414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:29:27.318646 26414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:29:27.318706 26414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:29:27.319818 26414 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:29:27.319890 26414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:29:27.319916 26414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:29:27.319942 26414 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:29:27.319984 26414 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:29:27.321523 26414 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790321110323796-5753-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39293
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790321110323796-5753-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26414
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:29:27.321789 26414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:29:27.322019 26414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:29:27.326462 26422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:27.326491 26419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:27.326476 26414 server_base.cc:1034] running on GCE node
W20260925 07:29:27.326453 26420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:27.326857 26414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:29:27.327147 26414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:29:27.328296 26414 hybrid_clock.cc:648] HybridClock initialized: now 1790321367328276 us; error 32 us; skew 500 ppm
Sep 25 07:29:27 dist-test-slave-1drq krb5kdc[26401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321367, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:29:27.330593 26414 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 07:29:27.330996 26414 webserver.cc:460] Webserver started at http://127.0.0.1:38583/ using document root <none> and password file <none>
I20260925 07:29:27.331172 26414 fs_manager.cc:359] Metadata directory not provided
I20260925 07:29:27.331209 26414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:29:27.331292 26414 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:29:27.332180 26414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790321110323796-5753-0/minicluster-data/master-0/data/instance:
uuid: "a3ac8da68d624c3086b7fd67e4ca9a9f"
format_stamp: "Formatted at 2026-09-25 07:29:27 on dist-test-slave-1drq"
server_key: "91b4101d62a7353e4fe454efaa9a9995"
server_key_iv: "bad027333fe876470d2c005be41952ec"
server_key_version: "encryptionkey@0"
I20260925 07:29:27.332435 26414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790321110323796-5753-0/minicluster-data/master-0/wal/instance:
uuid: "a3ac8da68d624c3086b7fd67e4ca9a9f"
format_stamp: "Formatted at 2026-09-25 07:29:27 on dist-test-slave-1drq"
server_key: "91b4101d62a7353e4fe454efaa9a9995"
server_key_iv: "bad027333fe876470d2c005be41952ec"
server_key_version: "encryptionkey@0"
I20260925 07:29:27.333618 26414 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260925 07:29:27.334285 26429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:27.334435 26414 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:29:27.334504 26414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "a3ac8da68d624c3086b7fd67e4ca9a9f"
format_stamp: "Formatted at 2026-09-25 07:29:27 on dist-test-slave-1drq"
server_key: "91b4101d62a7353e4fe454efaa9a9995"
server_key_iv: "bad027333fe876470d2c005be41952ec"
server_key_version: "encryptionkey@0"
I20260925 07:29:27.334550 26414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:29:27.347445 26414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:29:27.349375 26414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:29:27.349466 26414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:29:27.353065 26414 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39293
I20260925 07:29:27.353199 26481 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39293 every 8 connection(s)
I20260925 07:29:27.353443 26414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:29:27.354357 26482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:29:27.356273 26482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3ac8da68d624c3086b7fd67e4ca9a9f: Bootstrap starting.
I20260925 07:29:27.356760 26482 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a3ac8da68d624c3086b7fd67e4ca9a9f: Neither blocks nor log segments found. Creating new log.
I20260925 07:29:27.357036 26482 log.cc:824] T 00000000000000000000000000000000 P a3ac8da68d624c3086b7fd67e4ca9a9f: Log is configured to *not* fsync() on all Append() calls
I20260925 07:29:27.357715 26482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3ac8da68d624c3086b7fd67e4ca9a9f: No bootstrap required, opened a new log
I20260925 07:29:27.359198 26482 raft_consensus.cc:348] T 00000000000000000000000000000000 P a3ac8da68d624c3086b7fd67e4ca9a9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3ac8da68d624c3086b7fd67e4ca9a9f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39293 } }
I20260925 07:29:27.359340 26482 raft_consensus.cc:374] T 00000000000000000000000000000000 P a3ac8da68d624c3086b7fd67e4ca9a9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:29:27.359383 26482 raft_consensus.cc:729] T 00000000000000000000000000000000 P a3ac8da68d624c3086b7fd67e4ca9a9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3ac8da68d624c3086b7fd67e4ca9a9f, State: Initialized, Role: FOLLOWER
I20260925 07:29:27.359586 26482 consensus_queue.cc:260] T 00000000000000000000000000000000 P a3ac8da68d624c3086b7fd67e4ca9a9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3ac8da68d624c3086b7fd67e4ca9a9f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39293 } }
I20260925 07:29:27.359688 26482 raft_consensus.cc:388] T 00000000000000000000000000000000 P a3ac8da68d624c3086b7fd67e4ca9a9f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 07:29:27.359732 26482 raft_consensus.cc:482] T 00000000000000000000000000000000 P a3ac8da68d624c3086b7fd67e4ca9a9f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 07:29:27.359786 26482 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a3ac8da68d624c3086b7fd67e4ca9a9f [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:29:27.360285 26482 raft_consensus.cc:504] T 00000000000000000000000000000000 P a3ac8da68d624c3086b7fd67e4ca9a9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3ac8da68d624c3086b7fd67e4ca9a9f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39293 } }
I20260925 07:29:27.360414 26482 leader_election.cc:302] T 00000000000000000000000000000000 P a3ac8da68d624c3086b7fd67e4ca9a9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a3ac8da68d624c3086b7fd67e4ca9a9f; no voters:
I20260925 07:29:27.360594 26482 leader_election.cc:288] T 00000000000000000000000000000000 P a3ac8da68d624c3086b7fd67e4ca9a9f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 07:29:27.360620 26485 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a3ac8da68d624c3086b7fd67e4ca9a9f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:29:27.360759 26485 raft_consensus.cc:686] T 00000000000000000000000000000000 P a3ac8da68d624c3086b7fd67e4ca9a9f [term 1 LEADER]: Becoming Leader. State: Replica: a3ac8da68d624c3086b7fd67e4ca9a9f, State: Running, Role: LEADER
I20260925 07:29:27.360834 26482 sys_catalog.cc:564] T 00000000000000000000000000000000 P a3ac8da68d624c3086b7fd67e4ca9a9f [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:29:27.360885 26485 consensus_queue.cc:237] T 00000000000000000000000000000000 P a3ac8da68d624c3086b7fd67e4ca9a9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3ac8da68d624c3086b7fd67e4ca9a9f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39293 } }
I20260925 07:29:27.361445 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 26414
I20260925 07:29:27.361558 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790321110323796-5753-0/minicluster-data/master-0/wal/instance
I20260925 07:29:27.361747 5753 external_mini_cluster.cc:1428] Setting key bb9e3a37488d1f1465ce7ec580b0b3bf
I20260925 07:29:27.361804 26487 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3ac8da68d624c3086b7fd67e4ca9a9f [sys.catalog]: SysCatalogTable state changed. Reason: New leader a3ac8da68d624c3086b7fd67e4ca9a9f. Latest consensus state: current_term: 1 leader_uuid: "a3ac8da68d624c3086b7fd67e4ca9a9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3ac8da68d624c3086b7fd67e4ca9a9f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39293 } } }
I20260925 07:29:27.361897 26487 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3ac8da68d624c3086b7fd67e4ca9a9f [sys.catalog]: This master's current role is: LEADER
I20260925 07:29:27.361400 26486 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3ac8da68d624c3086b7fd67e4ca9a9f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a3ac8da68d624c3086b7fd67e4ca9a9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3ac8da68d624c3086b7fd67e4ca9a9f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39293 } } }
I20260925 07:29:27.362241 26486 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3ac8da68d624c3086b7fd67e4ca9a9f [sys.catalog]: This master's current role is: LEADER
I20260925 07:29:27.362648 26495 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 07:29:27.363229 26495 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 07:29:27.364590 26495 catalog_manager.cc:1347] Generated new cluster ID: 41d1c1e6672a4886bddb702248692471
I20260925 07:29:27.364638 26495 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 07:29:27.366848 26499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:27.362204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42109 (local address 127.0.0.1:39293)
0925 07:29:27.362351 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:27.362360 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:27.362544 (+ 184us) server_negotiation.cc:407] Connection header received
0925 07:29:27.362685 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:27.362691 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:27.362742 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:27.362865 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:27.363079 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:27.364097 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:27.365036 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:27.365511 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:27.365635 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:27.366235 (+ 600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:27.366238 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:27.366239 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:29:27.366408 (+ 169us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:27.366432 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:29:27.366442 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:27.366511 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:27.366574 (+ 63us) server_negotiation.cc:299] Negotiation successful
0925 07:29:27.366632 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":70,"threads_started":1}
I20260925 07:29:27.367553 5753 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 07:29:27.371541 26499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:27.368803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55925 (local address 127.0.0.1:39293)
0925 07:29:27.368839 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:27.368841 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:27.368970 (+ 129us) server_negotiation.cc:407] Connection header received
0925 07:29:27.369039 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:27.369039 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:27.369049 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:27.369139 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:27.369271 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:27.369723 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:27.370403 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:27.370615 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:27.370686 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:27.371292 (+ 606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:27.371293 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:27.371293 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:29:27.371297 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:27.371320 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:29:27.371324 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:27.371372 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:27.371435 (+ 63us) server_negotiation.cc:299] Negotiation successful
0925 07:29:27.371455 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 07:29:27.375456 26495 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 07:29:27.375911 26495 catalog_manager.cc:1504] Loading token signing keys...
I20260925 07:29:27.388350 26495 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a3ac8da68d624c3086b7fd67e4ca9a9f: Generated new TSK 0
I20260925 07:29:27.388569 26495 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 07:29:27.395306 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 26414
2026-09-25T07:29:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1187 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790321110323796-5753-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 07:29:27 dist-test-slave-1drq krb5kdc[26513](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 07:29:27 dist-test-slave-1drq krb5kdc[26513](info): set up 2 sockets
Sep 25 07:29:27 dist-test-slave-1drq krb5kdc[26513](info): commencing operation
krb5kdc: starting...
W20260925 07:29:28.336714 5753 mini_kdc.cc:121] Time spent starting KDC: real 0.924s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Sep 25 07:29:28 dist-test-slave-1drq krb5kdc[26513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321368, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T07:29:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T07:29:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790321110323796-5753-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790321110323796-5753-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 07:29:28.437985 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45281
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790321110323796-5753-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45281
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:29:28.511603 26526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:29:28.511811 26526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:29:28.511844 26526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:29:28.512928 26526 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:29:28.512991 26526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:29:28.513013 26526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:29:28.513034 26526 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:29:28.513059 26526 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:29:28.514416 26526 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790321110323796-5753-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45281
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790321110323796-5753-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26526
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:29:28.514672 26526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:29:28.514875 26526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:29:28.519289 26532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:28.519307 26534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:28.519289 26531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:28.519546 26526 server_base.cc:1034] running on GCE node
I20260925 07:29:28.519728 26526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:29:28.520007 26526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:29:28.521171 26526 hybrid_clock.cc:648] HybridClock initialized: now 1790321368521156 us; error 31 us; skew 500 ppm
Sep 25 07:29:28 dist-test-slave-1drq krb5kdc[26513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321368, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:29:28.526846 26526 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 07:29:28.527252 26526 webserver.cc:460] Webserver started at http://127.0.0.1:35631/ using document root <none> and password file <none>
I20260925 07:29:28.527451 26526 fs_manager.cc:359] Metadata directory not provided
I20260925 07:29:28.527511 26526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:29:28.527653 26526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:29:28.528538 26526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790321110323796-5753-0/minicluster-data/master-0/data/instance:
uuid: "9535b0e1dd5a4257834037709b76ed6e"
format_stamp: "Formatted at 2026-09-25 07:29:28 on dist-test-slave-1drq"
server_key: "ccbc5de97e11487d5a4f844af107631c"
server_key_iv: "e5817a4f47340a479087c917c6149c1c"
server_key_version: "encryptionkey@0"
I20260925 07:29:28.528865 26526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790321110323796-5753-0/minicluster-data/master-0/wal/instance:
uuid: "9535b0e1dd5a4257834037709b76ed6e"
format_stamp: "Formatted at 2026-09-25 07:29:28 on dist-test-slave-1drq"
server_key: "ccbc5de97e11487d5a4f844af107631c"
server_key_iv: "e5817a4f47340a479087c917c6149c1c"
server_key_version: "encryptionkey@0"
I20260925 07:29:28.530074 26526 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 07:29:28.530788 26541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:28.530954 26526 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:29:28.531046 26526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "9535b0e1dd5a4257834037709b76ed6e"
format_stamp: "Formatted at 2026-09-25 07:29:28 on dist-test-slave-1drq"
server_key: "ccbc5de97e11487d5a4f844af107631c"
server_key_iv: "e5817a4f47340a479087c917c6149c1c"
server_key_version: "encryptionkey@0"
I20260925 07:29:28.531140 26526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:29:28.534473 26526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:29:28.536777 26526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:29:28.536939 26526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:29:28.541006 26526 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45281
I20260925 07:29:28.541039 26593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45281 every 8 connection(s)
I20260925 07:29:28.541399 26526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:29:28.542237 26594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:29:28.544098 26594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9535b0e1dd5a4257834037709b76ed6e: Bootstrap starting.
I20260925 07:29:28.544649 26594 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9535b0e1dd5a4257834037709b76ed6e: Neither blocks nor log segments found. Creating new log.
I20260925 07:29:28.544905 26594 log.cc:824] T 00000000000000000000000000000000 P 9535b0e1dd5a4257834037709b76ed6e: Log is configured to *not* fsync() on all Append() calls
I20260925 07:29:28.545605 26594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9535b0e1dd5a4257834037709b76ed6e: No bootstrap required, opened a new log
I20260925 07:29:28.546993 26594 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9535b0e1dd5a4257834037709b76ed6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9535b0e1dd5a4257834037709b76ed6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45281 } }
I20260925 07:29:28.547099 26594 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9535b0e1dd5a4257834037709b76ed6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:29:28.547117 26594 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9535b0e1dd5a4257834037709b76ed6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9535b0e1dd5a4257834037709b76ed6e, State: Initialized, Role: FOLLOWER
I20260925 07:29:28.547290 26594 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9535b0e1dd5a4257834037709b76ed6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9535b0e1dd5a4257834037709b76ed6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45281 } }
I20260925 07:29:28.547346 26594 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9535b0e1dd5a4257834037709b76ed6e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 07:29:28.547365 26594 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9535b0e1dd5a4257834037709b76ed6e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 07:29:28.547408 26594 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9535b0e1dd5a4257834037709b76ed6e [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:29:28.547859 26594 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9535b0e1dd5a4257834037709b76ed6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9535b0e1dd5a4257834037709b76ed6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45281 } }
I20260925 07:29:28.547955 26594 leader_election.cc:302] T 00000000000000000000000000000000 P 9535b0e1dd5a4257834037709b76ed6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9535b0e1dd5a4257834037709b76ed6e; no voters:
I20260925 07:29:28.548120 26594 leader_election.cc:288] T 00000000000000000000000000000000 P 9535b0e1dd5a4257834037709b76ed6e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 07:29:28.548187 26597 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9535b0e1dd5a4257834037709b76ed6e [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:29:28.548374 26594 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9535b0e1dd5a4257834037709b76ed6e [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:29:28.548398 26597 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9535b0e1dd5a4257834037709b76ed6e [term 1 LEADER]: Becoming Leader. State: Replica: 9535b0e1dd5a4257834037709b76ed6e, State: Running, Role: LEADER
I20260925 07:29:28.548506 26597 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9535b0e1dd5a4257834037709b76ed6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9535b0e1dd5a4257834037709b76ed6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45281 } }
I20260925 07:29:28.549204 26599 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9535b0e1dd5a4257834037709b76ed6e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9535b0e1dd5a4257834037709b76ed6e. Latest consensus state: current_term: 1 leader_uuid: "9535b0e1dd5a4257834037709b76ed6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9535b0e1dd5a4257834037709b76ed6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45281 } } }
I20260925 07:29:28.549279 26599 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9535b0e1dd5a4257834037709b76ed6e [sys.catalog]: This master's current role is: LEADER
I20260925 07:29:28.549398 26598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9535b0e1dd5a4257834037709b76ed6e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9535b0e1dd5a4257834037709b76ed6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9535b0e1dd5a4257834037709b76ed6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45281 } } }
I20260925 07:29:28.549494 26598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9535b0e1dd5a4257834037709b76ed6e [sys.catalog]: This master's current role is: LEADER
I20260925 07:29:28.549904 26606 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 07:29:28.550568 26606 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 07:29:28.551666 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 26526
I20260925 07:29:28.551791 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790321110323796-5753-0/minicluster-data/master-0/wal/instance
I20260925 07:29:28.552033 5753 external_mini_cluster.cc:1428] Setting key e69677c3543b62577065ae60db2d4936
I20260925 07:29:28.552081 26606 catalog_manager.cc:1347] Generated new cluster ID: 474a12fca37147e4bee34fc74597dff2
I20260925 07:29:28.552165 26606 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 07:29:28.554214 26617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:28.552568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50435 (local address 127.0.0.1:45281)
0925 07:29:28.552731 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:28.552739 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:28.552774 (+ 35us) server_negotiation.cc:407] Connection header received
0925 07:29:28.552883 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:28.552889 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:28.552913 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:28.553031 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:28.553708 (+ 677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:28.553711 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:28.553712 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:29:28.553715 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:28.553763 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:29:28.553785 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:28.553847 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:28.553913 (+ 66us) server_negotiation.cc:299] Negotiation successful
0925 07:29:28.554028 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":95,"threads_started":1}
I20260925 07:29:28.555063 5753 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 07:29:28.557509 26617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:28.556651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40601 (local address 127.0.0.1:45281)
0925 07:29:28.556682 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:28.556683 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:28.556691 (+ 8us) server_negotiation.cc:407] Connection header received
0925 07:29:28.556708 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:28.556708 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:28.556711 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:28.556772 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:28.557303 (+ 531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:28.557304 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:28.557304 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:29:28.557308 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:28.557320 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:29:28.557324 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:28.557380 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:28.557446 (+ 66us) server_negotiation.cc:299] Negotiation successful
0925 07:29:28.557459 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 07:29:28.565749 26606 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 07:29:28.565882 26606 catalog_manager.cc:1504] Loading token signing keys...
I20260925 07:29:28.570465 26606 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9535b0e1dd5a4257834037709b76ed6e: Generated new TSK 0
I20260925 07:29:28.570715 26606 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 07:29:28.580428 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 26526
2026-09-25T07:29:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1185 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790321110323796-5753-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 07:29:28 dist-test-slave-1drq krb5kdc[26625](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 07:29:28 dist-test-slave-1drq krb5kdc[26625](info): set up 2 sockets
Sep 25 07:29:28 dist-test-slave-1drq krb5kdc[26625](info): commencing operation
krb5kdc: starting...
W20260925 07:29:29.528880 5753 mini_kdc.cc:121] Time spent starting KDC: real 0.931s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Sep 25 07:29:29 dist-test-slave-1drq krb5kdc[26625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T07:29:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T07:29:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790321110323796-5753-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790321110323796-5753-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 07:29:29.629882 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46835
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790321110323796-5753-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46835
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:29:29.705509 26638 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:29:29.705694 26638 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:29:29.705716 26638 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:29:29.707156 26638 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:29:29.707206 26638 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:29:29.707222 26638 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:29:29.707326 26638 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:29:29.707399 26638 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:29:29.708989 26638 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790321110323796-5753-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46835
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790321110323796-5753-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46835
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26638
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:29:29.709244 26638 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:29:29.709458 26638 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:29:29.713949 26646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:29.713953 26644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:29.713991 26643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:29.714176 26638 server_base.cc:1034] running on GCE node
I20260925 07:29:29.714414 26638 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:29:29.714687 26638 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:29:29.715822 26638 hybrid_clock.cc:648] HybridClock initialized: now 1790321369715810 us; error 31 us; skew 500 ppm
Sep 25 07:29:29 dist-test-slave-1drq krb5kdc[26625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321369, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:29:29.718152 26638 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 07:29:29.718549 26638 webserver.cc:460] Webserver started at http://127.0.0.1:34887/ using document root <none> and password file <none>
I20260925 07:29:29.718744 26638 fs_manager.cc:359] Metadata directory not provided
I20260925 07:29:29.718804 26638 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:29:29.718940 26638 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:29:29.719893 26638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790321110323796-5753-0/minicluster-data/master-0/data/instance:
uuid: "dd29cf1138434a3998dddf91ba7eac67"
format_stamp: "Formatted at 2026-09-25 07:29:29 on dist-test-slave-1drq"
server_key: "b138cc02a4e27354319a482097707116"
server_key_iv: "d808ac2a171c0716b89589b0d5d1bf3a"
server_key_version: "encryptionkey@0"
I20260925 07:29:29.720239 26638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790321110323796-5753-0/minicluster-data/master-0/wal/instance:
uuid: "dd29cf1138434a3998dddf91ba7eac67"
format_stamp: "Formatted at 2026-09-25 07:29:29 on dist-test-slave-1drq"
server_key: "b138cc02a4e27354319a482097707116"
server_key_iv: "d808ac2a171c0716b89589b0d5d1bf3a"
server_key_version: "encryptionkey@0"
I20260925 07:29:29.721668 26638 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260925 07:29:29.722374 26653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:29.722535 26638 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:29:29.722651 26638 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "dd29cf1138434a3998dddf91ba7eac67"
format_stamp: "Formatted at 2026-09-25 07:29:29 on dist-test-slave-1drq"
server_key: "b138cc02a4e27354319a482097707116"
server_key_iv: "d808ac2a171c0716b89589b0d5d1bf3a"
server_key_version: "encryptionkey@0"
I20260925 07:29:29.722720 26638 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:29:29.738638 26638 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:29:29.740170 26638 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:29:29.740319 26638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:29:29.744021 26638 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46835
I20260925 07:29:29.744052 26705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46835 every 8 connection(s)
I20260925 07:29:29.744346 26638 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:29:29.745182 26706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:29:29.747017 26706 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd29cf1138434a3998dddf91ba7eac67: Bootstrap starting.
I20260925 07:29:29.747488 26706 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd29cf1138434a3998dddf91ba7eac67: Neither blocks nor log segments found. Creating new log.
I20260925 07:29:29.747704 26706 log.cc:824] T 00000000000000000000000000000000 P dd29cf1138434a3998dddf91ba7eac67: Log is configured to *not* fsync() on all Append() calls
I20260925 07:29:29.748274 26706 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd29cf1138434a3998dddf91ba7eac67: No bootstrap required, opened a new log
I20260925 07:29:29.749727 26706 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd29cf1138434a3998dddf91ba7eac67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd29cf1138434a3998dddf91ba7eac67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46835 } }
I20260925 07:29:29.749845 26706 raft_consensus.cc:374] T 00000000000000000000000000000000 P dd29cf1138434a3998dddf91ba7eac67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:29:29.749866 26706 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd29cf1138434a3998dddf91ba7eac67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd29cf1138434a3998dddf91ba7eac67, State: Initialized, Role: FOLLOWER
I20260925 07:29:29.749951 26706 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd29cf1138434a3998dddf91ba7eac67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd29cf1138434a3998dddf91ba7eac67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46835 } }
I20260925 07:29:29.749998 26706 raft_consensus.cc:388] T 00000000000000000000000000000000 P dd29cf1138434a3998dddf91ba7eac67 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 07:29:29.750018 26706 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd29cf1138434a3998dddf91ba7eac67 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 07:29:29.750042 26706 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd29cf1138434a3998dddf91ba7eac67 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:29:29.750484 26706 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd29cf1138434a3998dddf91ba7eac67 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd29cf1138434a3998dddf91ba7eac67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46835 } }
I20260925 07:29:29.750561 26706 leader_election.cc:302] T 00000000000000000000000000000000 P dd29cf1138434a3998dddf91ba7eac67 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd29cf1138434a3998dddf91ba7eac67; no voters:
I20260925 07:29:29.750667 26706 leader_election.cc:288] T 00000000000000000000000000000000 P dd29cf1138434a3998dddf91ba7eac67 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 07:29:29.750718 26709 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd29cf1138434a3998dddf91ba7eac67 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:29:29.750909 26706 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd29cf1138434a3998dddf91ba7eac67 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:29:29.750943 26709 raft_consensus.cc:686] T 00000000000000000000000000000000 P dd29cf1138434a3998dddf91ba7eac67 [term 1 LEADER]: Becoming Leader. State: Replica: dd29cf1138434a3998dddf91ba7eac67, State: Running, Role: LEADER
I20260925 07:29:29.751026 26709 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd29cf1138434a3998dddf91ba7eac67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd29cf1138434a3998dddf91ba7eac67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46835 } }
I20260925 07:29:29.751601 26710 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd29cf1138434a3998dddf91ba7eac67 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dd29cf1138434a3998dddf91ba7eac67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd29cf1138434a3998dddf91ba7eac67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46835 } } }
I20260925 07:29:29.751616 26711 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd29cf1138434a3998dddf91ba7eac67 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd29cf1138434a3998dddf91ba7eac67. Latest consensus state: current_term: 1 leader_uuid: "dd29cf1138434a3998dddf91ba7eac67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd29cf1138434a3998dddf91ba7eac67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46835 } } }
I20260925 07:29:29.751696 26710 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd29cf1138434a3998dddf91ba7eac67 [sys.catalog]: This master's current role is: LEADER
I20260925 07:29:29.751710 26711 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd29cf1138434a3998dddf91ba7eac67 [sys.catalog]: This master's current role is: LEADER
I20260925 07:29:29.751873 26718 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 07:29:29.752342 26718 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 07:29:29.753574 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 26638
I20260925 07:29:29.753679 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790321110323796-5753-0/minicluster-data/master-0/wal/instance
I20260925 07:29:29.753724 26718 catalog_manager.cc:1347] Generated new cluster ID: ff8d911cf6ef4aa48fbfa2c957936272
I20260925 07:29:29.753760 26718 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 07:29:29.753870 5753 external_mini_cluster.cc:1428] Setting key 9b12e6288ec8597e1bb0620abd5a5b3c
Sep 25 07:29:29 dist-test-slave-1drq krb5kdc[26625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260925 07:29:29.769237 26729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:29.757431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.72:47615 (local address 127.0.0.1:46835)
0925 07:29:29.757589 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:29.757597 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:29.761325 (+ 3728us) server_negotiation.cc:407] Connection header received
0925 07:29:29.761469 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:29.761474 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:29.761515 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:29.761645 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:29.762190 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:29.762990 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:29.763672 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:29.764084 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:29.764194 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:29.766407 (+ 2213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:29.766411 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:29.766411 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:29:29.766421 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:29.767865 (+ 1444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:29.768212 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:29.768215 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:29.768215 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:29.768298 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:29.768505 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:29.768506 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:29.768506 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:29.768712 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:29.768873 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:29.768972 (+ 99us) server_negotiation.cc:299] Negotiation successful
0925 07:29:29.769061 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":77,"threads_started":1}
I20260925 07:29:29.770071 5753 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 07:29:29.779139 26718 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 07:29:29.779778 26718 catalog_manager.cc:1504] Loading token signing keys...
I20260925 07:29:29.786597 26729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:29.778906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.72:45617 (local address 127.0.0.1:46835)
0925 07:29:29.779048 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:29.779050 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:29.779058 (+ 8us) server_negotiation.cc:407] Connection header received
0925 07:29:29.779075 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:29.779076 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:29.779084 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:29.779162 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:29.780070 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:29.781056 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:29.782481 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:29.782922 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:29.783013 (+ 91us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:29.784084 (+ 1071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:29.784085 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:29.784086 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:29:29.784092 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:29.785402 (+ 1310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:29.785739 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:29.785740 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:29.785740 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:29.785788 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:29.786017 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:29.786018 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:29.786019 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:29.786120 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:29.786235 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:29.786342 (+ 107us) server_negotiation.cc:299] Negotiation successful
0925 07:29:29.786502 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":9,"server-negotiator.queue_time_us":106}
I20260925 07:29:29.791188 26718 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dd29cf1138434a3998dddf91ba7eac67: Generated new TSK 0
I20260925 07:29:29.791356 26718 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 07:29:29.794204 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 26638
W20260925 07:29:29.799521 26623 connection.cc:535] client connection to 127.0.0.1:46835 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46835): Connection reset by peer (error 104)
2026-09-25T07:29:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1215 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790321110323796-5753-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 07:29:29 dist-test-slave-1drq krb5kdc[26737](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 07:29:29 dist-test-slave-1drq krb5kdc[26737](info): set up 2 sockets
Sep 25 07:29:29 dist-test-slave-1drq krb5kdc[26737](info): commencing operation
krb5kdc: starting...
W20260925 07:29:30.732921 5753 mini_kdc.cc:121] Time spent starting KDC: real 0.920s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Sep 25 07:29:30 dist-test-slave-1drq krb5kdc[26737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321370, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T07:29:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T07:29:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790321110323796-5753-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790321110323796-5753-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 07:29:30.833510 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40071
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790321110323796-5753-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40071
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:29:30.907485 26750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:29:30.907666 26750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:29:30.907686 26750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:29:30.908704 26750 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:29:30.908741 26750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:29:30.908754 26750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:29:30.908799 26750 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:29:30.908825 26750 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:29:30.910228 26750 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790321110323796-5753-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40071
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790321110323796-5753-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26750
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:29:30.910465 26750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:29:30.910676 26750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 07:29:30.915016 26750 server_base.cc:1034] running on GCE node
W20260925 07:29:30.914995 26758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:30.915001 26756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:30.915007 26755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:30.915388 26750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:29:30.915683 26750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:29:30.916829 26750 hybrid_clock.cc:648] HybridClock initialized: now 1790321370916819 us; error 39 us; skew 500 ppm
Sep 25 07:29:30 dist-test-slave-1drq krb5kdc[26737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321370, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:29:30.918965 26750 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 07:29:30.919261 26750 webserver.cc:460] Webserver started at http://127.0.0.1:44343/ using document root <none> and password file <none>
I20260925 07:29:30.919416 26750 fs_manager.cc:359] Metadata directory not provided
I20260925 07:29:30.919449 26750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:29:30.919520 26750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:29:30.920362 26750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790321110323796-5753-0/minicluster-data/master-0/data/instance:
uuid: "0dd6b7e069324ffc865782dbf5e54402"
format_stamp: "Formatted at 2026-09-25 07:29:30 on dist-test-slave-1drq"
server_key: "1da53c9abefeaf3b517e94bb00cc0558"
server_key_iv: "2906588459b86b8bb3b31ba4b3eb95b6"
server_key_version: "encryptionkey@0"
I20260925 07:29:30.920631 26750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790321110323796-5753-0/minicluster-data/master-0/wal/instance:
uuid: "0dd6b7e069324ffc865782dbf5e54402"
format_stamp: "Formatted at 2026-09-25 07:29:30 on dist-test-slave-1drq"
server_key: "1da53c9abefeaf3b517e94bb00cc0558"
server_key_iv: "2906588459b86b8bb3b31ba4b3eb95b6"
server_key_version: "encryptionkey@0"
I20260925 07:29:30.921896 26750 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260925 07:29:30.922494 26765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:30.922644 26750 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:30.922705 26750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "0dd6b7e069324ffc865782dbf5e54402"
format_stamp: "Formatted at 2026-09-25 07:29:30 on dist-test-slave-1drq"
server_key: "1da53c9abefeaf3b517e94bb00cc0558"
server_key_iv: "2906588459b86b8bb3b31ba4b3eb95b6"
server_key_version: "encryptionkey@0"
I20260925 07:29:30.922748 26750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:29:30.930624 26750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:29:30.932120 26750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:29:30.932207 26750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:29:30.935653 26750 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40071
I20260925 07:29:30.935696 26817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40071 every 8 connection(s)
I20260925 07:29:30.935961 26750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:29:30.936775 26818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:29:30.937004 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 26750
I20260925 07:29:30.937162 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790321110323796-5753-0/minicluster-data/master-0/wal/instance
I20260925 07:29:30.937359 5753 external_mini_cluster.cc:1428] Setting key 378f16b094d485117b54be912ae62f72
I20260925 07:29:30.939270 26818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dd6b7e069324ffc865782dbf5e54402: Bootstrap starting.
I20260925 07:29:30.939838 26818 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0dd6b7e069324ffc865782dbf5e54402: Neither blocks nor log segments found. Creating new log.
I20260925 07:29:30.940140 26818 log.cc:824] T 00000000000000000000000000000000 P 0dd6b7e069324ffc865782dbf5e54402: Log is configured to *not* fsync() on all Append() calls
I20260925 07:29:30.940809 26818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dd6b7e069324ffc865782dbf5e54402: No bootstrap required, opened a new log
I20260925 07:29:30.942340 26821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:30.937814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.72:43719 (local address 127.0.0.1:40071)
0925 07:29:30.937970 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:30.937978 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:30.938009 (+ 31us) server_negotiation.cc:407] Connection header received
0925 07:29:30.938358 (+ 349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:30.938363 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:30.938403 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:30.938505 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:30.938757 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:30.939591 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:30.940642 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:30.941054 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:30.941199 (+ 145us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:30.941796 (+ 597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:30.941800 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:30.941800 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:29:30.941929 (+ 129us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:30.941953 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:29:30.941961 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:30.942018 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:30.942126 (+ 108us) server_negotiation.cc:299] Negotiation successful
0925 07:29:30.942164 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":70,"threads_started":1}
I20260925 07:29:30.943006 26818 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0dd6b7e069324ffc865782dbf5e54402 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dd6b7e069324ffc865782dbf5e54402" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40071 } }
I20260925 07:29:30.943192 26818 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0dd6b7e069324ffc865782dbf5e54402 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:29:30.943238 26818 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0dd6b7e069324ffc865782dbf5e54402 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dd6b7e069324ffc865782dbf5e54402, State: Initialized, Role: FOLLOWER
I20260925 07:29:30.943346 26818 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0dd6b7e069324ffc865782dbf5e54402 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dd6b7e069324ffc865782dbf5e54402" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40071 } }
I20260925 07:29:30.943431 26818 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0dd6b7e069324ffc865782dbf5e54402 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 07:29:30.943435 5753 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 07:29:30.943460 26818 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0dd6b7e069324ffc865782dbf5e54402 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 07:29:30.943502 26818 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0dd6b7e069324ffc865782dbf5e54402 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:29:30.943950 26818 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0dd6b7e069324ffc865782dbf5e54402 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dd6b7e069324ffc865782dbf5e54402" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40071 } }
I20260925 07:29:30.944021 26818 leader_election.cc:302] T 00000000000000000000000000000000 P 0dd6b7e069324ffc865782dbf5e54402 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0dd6b7e069324ffc865782dbf5e54402; no voters:
I20260925 07:29:30.944178 26818 leader_election.cc:288] T 00000000000000000000000000000000 P 0dd6b7e069324ffc865782dbf5e54402 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 07:29:30.944202 26823 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0dd6b7e069324ffc865782dbf5e54402 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:29:30.944288 26823 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0dd6b7e069324ffc865782dbf5e54402 [term 1 LEADER]: Becoming Leader. State: Replica: 0dd6b7e069324ffc865782dbf5e54402, State: Running, Role: LEADER
I20260925 07:29:30.944384 26823 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0dd6b7e069324ffc865782dbf5e54402 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dd6b7e069324ffc865782dbf5e54402" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40071 } }
I20260925 07:29:30.944697 26823 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dd6b7e069324ffc865782dbf5e54402 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0dd6b7e069324ffc865782dbf5e54402" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dd6b7e069324ffc865782dbf5e54402" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40071 } } }
I20260925 07:29:30.944798 26823 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dd6b7e069324ffc865782dbf5e54402 [sys.catalog]: This master's current role is: LEADER
I20260925 07:29:30.944931 26823 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dd6b7e069324ffc865782dbf5e54402 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0dd6b7e069324ffc865782dbf5e54402. Latest consensus state: current_term: 1 leader_uuid: "0dd6b7e069324ffc865782dbf5e54402" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dd6b7e069324ffc865782dbf5e54402" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40071 } } }
I20260925 07:29:30.944990 26823 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dd6b7e069324ffc865782dbf5e54402 [sys.catalog]: This master's current role is: LEADER
I20260925 07:29:30.944381 26818 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0dd6b7e069324ffc865782dbf5e54402 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:29:30.945004 26832 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 07:29:30.945518 26832 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 07:29:30.948998 26821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:30.944680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.72:50149 (local address 127.0.0.1:40071)
0925 07:29:30.944709 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:30.944710 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:30.946348 (+ 1638us) server_negotiation.cc:407] Connection header received
0925 07:29:30.946403 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:30.946404 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:30.946414 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:30.946470 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:30.946655 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:30.947267 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:30.947876 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:30.948159 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:30.948227 (+ 68us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:30.948798 (+ 571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:30.948799 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:30.948799 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:29:30.948803 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:30.948815 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:29:30.948819 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:30.948885 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:30.948929 (+ 44us) server_negotiation.cc:299] Negotiation successful
0925 07:29:30.948940 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260925 07:29:30.949492 26832 catalog_manager.cc:1347] Generated new cluster ID: 618af3efa4604510bf55682acabcf8c5
I20260925 07:29:30.949537 26832 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 07:29:30.971346 26832 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 07:29:30.971836 26832 catalog_manager.cc:1504] Loading token signing keys...
I20260925 07:29:30.976202 26832 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0dd6b7e069324ffc865782dbf5e54402: Generated new TSK 0
I20260925 07:29:30.976351 26832 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 07:29:30.986364 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 26750
W20260925 07:29:30.989815 26735 connection.cc:535] client connection to 127.0.0.1:40071 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40071): Connection reset by peer (error 104)
2026-09-25T07:29:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1190 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790321110323796-5753-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 07:29:31 dist-test-slave-1drq krb5kdc[26849](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 07:29:31 dist-test-slave-1drq krb5kdc[26849](info): set up 2 sockets
Sep 25 07:29:31 dist-test-slave-1drq krb5kdc[26849](info): commencing operation
krb5kdc: starting...
W20260925 07:29:31.956658 5753 mini_kdc.cc:121] Time spent starting KDC: real 0.953s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Sep 25 07:29:32 dist-test-slave-1drq krb5kdc[26849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321372, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T07:29:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T07:29:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790321110323796-5753-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790321110323796-5753-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 07:29:32.060196 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34053
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790321110323796-5753-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34053
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:29:32.131852 26862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:29:32.132041 26862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:29:32.132076 26862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:29:32.133100 26862 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:29:32.133157 26862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:29:32.133181 26862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:29:32.133214 26862 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:29:32.133237 26862 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:29:32.134531 26862 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790321110323796-5753-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34053
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790321110323796-5753-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34053
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26862
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:29:32.134745 26862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:29:32.134966 26862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:29:32.139513 26868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:32.139539 26867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:32.139556 26862 server_base.cc:1034] running on GCE node
W20260925 07:29:32.139521 26870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:32.139940 26862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:29:32.140208 26862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:29:32.141360 26862 hybrid_clock.cc:648] HybridClock initialized: now 1790321372141339 us; error 48 us; skew 500 ppm
Sep 25 07:29:32 dist-test-slave-1drq krb5kdc[26849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321372, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:29:32.143451 26862 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 07:29:32.143877 26862 webserver.cc:460] Webserver started at http://127.0.0.1:41165/ using document root <none> and password file <none>
I20260925 07:29:32.144083 26862 fs_manager.cc:359] Metadata directory not provided
I20260925 07:29:32.144145 26862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:29:32.144275 26862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:29:32.145258 26862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790321110323796-5753-0/minicluster-data/master-0/data/instance:
uuid: "9b2beeec778649878e3f398787cc0276"
format_stamp: "Formatted at 2026-09-25 07:29:32 on dist-test-slave-1drq"
server_key: "d8cd7cf88a328e3c4f292379a6cba9b3"
server_key_iv: "083009bd49298d2a81f1f5fd8f45c914"
server_key_version: "encryptionkey@0"
I20260925 07:29:32.145571 26862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790321110323796-5753-0/minicluster-data/master-0/wal/instance:
uuid: "9b2beeec778649878e3f398787cc0276"
format_stamp: "Formatted at 2026-09-25 07:29:32 on dist-test-slave-1drq"
server_key: "d8cd7cf88a328e3c4f292379a6cba9b3"
server_key_iv: "083009bd49298d2a81f1f5fd8f45c914"
server_key_version: "encryptionkey@0"
I20260925 07:29:32.146716 26862 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 07:29:32.147343 26877 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:32.147491 26862 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:29:32.147589 26862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "9b2beeec778649878e3f398787cc0276"
format_stamp: "Formatted at 2026-09-25 07:29:32 on dist-test-slave-1drq"
server_key: "d8cd7cf88a328e3c4f292379a6cba9b3"
server_key_iv: "083009bd49298d2a81f1f5fd8f45c914"
server_key_version: "encryptionkey@0"
I20260925 07:29:32.147655 26862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:29:32.150734 26862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:29:32.152380 26862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:29:32.152532 26862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:29:32.156291 26862 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34053
I20260925 07:29:32.156318 26929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34053 every 8 connection(s)
I20260925 07:29:32.156603 26862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:29:32.157580 26930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:29:32.159468 26930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b2beeec778649878e3f398787cc0276: Bootstrap starting.
I20260925 07:29:32.159924 26930 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9b2beeec778649878e3f398787cc0276: Neither blocks nor log segments found. Creating new log.
I20260925 07:29:32.160159 26930 log.cc:824] T 00000000000000000000000000000000 P 9b2beeec778649878e3f398787cc0276: Log is configured to *not* fsync() on all Append() calls
I20260925 07:29:32.160766 26930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b2beeec778649878e3f398787cc0276: No bootstrap required, opened a new log
I20260925 07:29:32.162391 26930 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b2beeec778649878e3f398787cc0276 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b2beeec778649878e3f398787cc0276" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34053 } }
I20260925 07:29:32.162515 26930 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9b2beeec778649878e3f398787cc0276 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:29:32.162535 26930 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b2beeec778649878e3f398787cc0276 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b2beeec778649878e3f398787cc0276, State: Initialized, Role: FOLLOWER
I20260925 07:29:32.162621 26930 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b2beeec778649878e3f398787cc0276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b2beeec778649878e3f398787cc0276" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34053 } }
I20260925 07:29:32.162672 26930 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9b2beeec778649878e3f398787cc0276 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 07:29:32.162690 26930 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b2beeec778649878e3f398787cc0276 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 07:29:32.162717 26930 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b2beeec778649878e3f398787cc0276 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:29:32.163113 26930 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b2beeec778649878e3f398787cc0276 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b2beeec778649878e3f398787cc0276" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34053 } }
I20260925 07:29:32.163183 26930 leader_election.cc:302] T 00000000000000000000000000000000 P 9b2beeec778649878e3f398787cc0276 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9b2beeec778649878e3f398787cc0276; no voters:
I20260925 07:29:32.163309 26930 leader_election.cc:288] T 00000000000000000000000000000000 P 9b2beeec778649878e3f398787cc0276 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 07:29:32.163336 26933 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b2beeec778649878e3f398787cc0276 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:29:32.163437 26933 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9b2beeec778649878e3f398787cc0276 [term 1 LEADER]: Becoming Leader. State: Replica: 9b2beeec778649878e3f398787cc0276, State: Running, Role: LEADER
I20260925 07:29:32.163532 26930 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b2beeec778649878e3f398787cc0276 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:29:32.163560 26933 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b2beeec778649878e3f398787cc0276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b2beeec778649878e3f398787cc0276" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34053 } }
I20260925 07:29:32.163743 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 26862
I20260925 07:29:32.163880 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790321110323796-5753-0/minicluster-data/master-0/wal/instance
I20260925 07:29:32.163978 26935 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b2beeec778649878e3f398787cc0276 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b2beeec778649878e3f398787cc0276. Latest consensus state: current_term: 1 leader_uuid: "9b2beeec778649878e3f398787cc0276" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b2beeec778649878e3f398787cc0276" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34053 } } }
I20260925 07:29:32.164044 26934 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b2beeec778649878e3f398787cc0276 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9b2beeec778649878e3f398787cc0276" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b2beeec778649878e3f398787cc0276" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34053 } } }
I20260925 07:29:32.164083 5753 external_mini_cluster.cc:1428] Setting key f2e756d2a018a416650309538ce18399
I20260925 07:29:32.164114 26935 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b2beeec778649878e3f398787cc0276 [sys.catalog]: This master's current role is: LEADER
I20260925 07:29:32.164170 26934 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b2beeec778649878e3f398787cc0276 [sys.catalog]: This master's current role is: LEADER
I20260925 07:29:32.164763 26937 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 07:29:32.165639 26937 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 07:29:32.166028 26941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:32.164517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.72:59503 (local address 127.0.0.1:34053)
0925 07:29:32.164701 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:32.164709 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:32.164735 (+ 26us) server_negotiation.cc:407] Connection header received
0925 07:29:32.164837 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:32.164842 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:32.164868 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:32.165002 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:32.165671 (+ 669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:32.165674 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:32.165675 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:29:32.165678 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:32.165718 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:29:32.165731 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:32.165787 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:32.165857 (+ 70us) server_negotiation.cc:299] Negotiation successful
0925 07:29:32.165894 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":72,"threads_started":1}
I20260925 07:29:32.166936 5753 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 07:29:32.167039 26937 catalog_manager.cc:1347] Generated new cluster ID: 93b8f1b95d274c8b9808fdd9fc87b435
I20260925 07:29:32.167086 26937 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 07:29:32.169987 26941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:32.169171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.72:59783 (local address 127.0.0.1:34053)
0925 07:29:32.169199 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:32.169201 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:32.169208 (+ 7us) server_negotiation.cc:407] Connection header received
0925 07:29:32.169224 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:32.169225 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:32.169228 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:32.169272 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:32.169774 (+ 502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:32.169775 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:32.169775 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:29:32.169779 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:32.169789 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:29:32.169793 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:32.169840 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:32.169905 (+ 65us) server_negotiation.cc:299] Negotiation successful
0925 07:29:32.169918 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
I20260925 07:29:32.173472 26937 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 07:29:32.173597 26937 catalog_manager.cc:1504] Loading token signing keys...
I20260925 07:29:32.177166 26937 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9b2beeec778649878e3f398787cc0276: Generated new TSK 0
I20260925 07:29:32.177302 26937 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 07:29:32.179004 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 26862
2026-09-25T07:29:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1192 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790321110323796-5753-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 07:29:32 dist-test-slave-1drq krb5kdc[26961](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 07:29:32 dist-test-slave-1drq krb5kdc[26961](info): set up 2 sockets
Sep 25 07:29:32 dist-test-slave-1drq krb5kdc[26961](info): commencing operation
krb5kdc: starting...
W20260925 07:29:33.116618 5753 mini_kdc.cc:121] Time spent starting KDC: real 0.921s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Sep 25 07:29:33 dist-test-slave-1drq krb5kdc[26961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T07:29:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T07:29:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790321110323796-5753-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790321110323796-5753-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 07:29:33.217013 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37225
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790321110323796-5753-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37225
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:29:33.289758 26974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:29:33.289925 26974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:29:33.289944 26974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:29:33.291081 26974 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:29:33.291121 26974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:29:33.291136 26974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:29:33.291148 26974 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:29:33.291160 26974 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:29:33.292544 26974 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790321110323796-5753-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37225
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790321110323796-5753-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37225
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26974
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:29:33.292739 26974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:29:33.292949 26974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:29:33.297220 26979 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:33.297202 26980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:33.297277 26974 server_base.cc:1034] running on GCE node
W20260925 07:29:33.297220 26982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:33.297664 26974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:29:33.297927 26974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:29:33.299074 26974 hybrid_clock.cc:648] HybridClock initialized: now 1790321373299074 us; error 36 us; skew 500 ppm
Sep 25 07:29:33 dist-test-slave-1drq krb5kdc[26961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321373, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:29:33.301445 26974 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 07:29:33.301789 26974 webserver.cc:460] Webserver started at http://127.0.0.1:36747/ using document root <none> and password file <none>
I20260925 07:29:33.301986 26974 fs_manager.cc:359] Metadata directory not provided
I20260925 07:29:33.302021 26974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:29:33.302094 26974 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:29:33.302912 26974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790321110323796-5753-0/minicluster-data/master-0/data/instance:
uuid: "e841e9bd927a422eac3b3d6e359b4c0a"
format_stamp: "Formatted at 2026-09-25 07:29:33 on dist-test-slave-1drq"
server_key: "0782c0886775004cefb83727f7dcfb2e"
server_key_iv: "8a57bb83b4694732eb2b28e8460e8173"
server_key_version: "encryptionkey@0"
I20260925 07:29:33.303157 26974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790321110323796-5753-0/minicluster-data/master-0/wal/instance:
uuid: "e841e9bd927a422eac3b3d6e359b4c0a"
format_stamp: "Formatted at 2026-09-25 07:29:33 on dist-test-slave-1drq"
server_key: "0782c0886775004cefb83727f7dcfb2e"
server_key_iv: "8a57bb83b4694732eb2b28e8460e8173"
server_key_version: "encryptionkey@0"
I20260925 07:29:33.304194 26974 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260925 07:29:33.304746 26989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:33.304891 26974 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:33.304965 26974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "e841e9bd927a422eac3b3d6e359b4c0a"
format_stamp: "Formatted at 2026-09-25 07:29:33 on dist-test-slave-1drq"
server_key: "0782c0886775004cefb83727f7dcfb2e"
server_key_iv: "8a57bb83b4694732eb2b28e8460e8173"
server_key_version: "encryptionkey@0"
I20260925 07:29:33.305011 26974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:29:33.314177 26974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:29:33.315739 26974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:29:33.315847 26974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:29:33.319391 26974 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37225
I20260925 07:29:33.319442 27041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37225 every 8 connection(s)
I20260925 07:29:33.319725 26974 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:29:33.320370 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 26974
I20260925 07:29:33.320518 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790321110323796-5753-0/minicluster-data/master-0/wal/instance
I20260925 07:29:33.320731 5753 external_mini_cluster.cc:1428] Setting key 2da8eaa24d5f2a66c5921d0dddf6d104
I20260925 07:29:33.320868 27042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:29:33.323392 27042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e841e9bd927a422eac3b3d6e359b4c0a: Bootstrap starting.
I20260925 07:29:33.323935 27042 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e841e9bd927a422eac3b3d6e359b4c0a: Neither blocks nor log segments found. Creating new log.
I20260925 07:29:33.324221 27042 log.cc:824] T 00000000000000000000000000000000 P e841e9bd927a422eac3b3d6e359b4c0a: Log is configured to *not* fsync() on all Append() calls
I20260925 07:29:33.324836 27042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e841e9bd927a422eac3b3d6e359b4c0a: No bootstrap required, opened a new log
Sep 25 07:29:33 dist-test-slave-1drq krb5kdc[26961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260925 07:29:33.326787 27042 raft_consensus.cc:348] T 00000000000000000000000000000000 P e841e9bd927a422eac3b3d6e359b4c0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e841e9bd927a422eac3b3d6e359b4c0a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37225 } }
I20260925 07:29:33.326926 27042 raft_consensus.cc:374] T 00000000000000000000000000000000 P e841e9bd927a422eac3b3d6e359b4c0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:29:33.326973 27042 raft_consensus.cc:729] T 00000000000000000000000000000000 P e841e9bd927a422eac3b3d6e359b4c0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e841e9bd927a422eac3b3d6e359b4c0a, State: Initialized, Role: FOLLOWER
I20260925 07:29:33.327095 27042 consensus_queue.cc:260] T 00000000000000000000000000000000 P e841e9bd927a422eac3b3d6e359b4c0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e841e9bd927a422eac3b3d6e359b4c0a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37225 } }
I20260925 07:29:33.327175 27042 raft_consensus.cc:388] T 00000000000000000000000000000000 P e841e9bd927a422eac3b3d6e359b4c0a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 07:29:33.327198 27042 raft_consensus.cc:482] T 00000000000000000000000000000000 P e841e9bd927a422eac3b3d6e359b4c0a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 07:29:33.327260 27042 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e841e9bd927a422eac3b3d6e359b4c0a [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:29:33.327752 27042 raft_consensus.cc:504] T 00000000000000000000000000000000 P e841e9bd927a422eac3b3d6e359b4c0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e841e9bd927a422eac3b3d6e359b4c0a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37225 } }
I20260925 07:29:33.327865 27042 leader_election.cc:302] T 00000000000000000000000000000000 P e841e9bd927a422eac3b3d6e359b4c0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e841e9bd927a422eac3b3d6e359b4c0a; no voters:
I20260925 07:29:33.328015 27042 leader_election.cc:288] T 00000000000000000000000000000000 P e841e9bd927a422eac3b3d6e359b4c0a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 07:29:33.328051 27047 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e841e9bd927a422eac3b3d6e359b4c0a [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:29:33.328274 27042 sys_catalog.cc:564] T 00000000000000000000000000000000 P e841e9bd927a422eac3b3d6e359b4c0a [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:29:33.328455 27047 raft_consensus.cc:686] T 00000000000000000000000000000000 P e841e9bd927a422eac3b3d6e359b4c0a [term 1 LEADER]: Becoming Leader. State: Replica: e841e9bd927a422eac3b3d6e359b4c0a, State: Running, Role: LEADER
I20260925 07:29:33.328553 27047 consensus_queue.cc:237] T 00000000000000000000000000000000 P e841e9bd927a422eac3b3d6e359b4c0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e841e9bd927a422eac3b3d6e359b4c0a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37225 } }
I20260925 07:29:33.328931 27047 sys_catalog.cc:455] T 00000000000000000000000000000000 P e841e9bd927a422eac3b3d6e359b4c0a [sys.catalog]: SysCatalogTable state changed. Reason: New leader e841e9bd927a422eac3b3d6e359b4c0a. Latest consensus state: current_term: 1 leader_uuid: "e841e9bd927a422eac3b3d6e359b4c0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e841e9bd927a422eac3b3d6e359b4c0a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37225 } } }
I20260925 07:29:33.328938 27049 sys_catalog.cc:455] T 00000000000000000000000000000000 P e841e9bd927a422eac3b3d6e359b4c0a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e841e9bd927a422eac3b3d6e359b4c0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e841e9bd927a422eac3b3d6e359b4c0a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37225 } } }
I20260925 07:29:33.329054 27049 sys_catalog.cc:458] T 00000000000000000000000000000000 P e841e9bd927a422eac3b3d6e359b4c0a [sys.catalog]: This master's current role is: LEADER
I20260925 07:29:33.329254 27047 sys_catalog.cc:458] T 00000000000000000000000000000000 P e841e9bd927a422eac3b3d6e359b4c0a [sys.catalog]: This master's current role is: LEADER
I20260925 07:29:33.329422 27056 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 07:29:33.330140 27056 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 07:29:33.331172 27045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:33.321210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.72:47969 (local address 127.0.0.1:37225)
0925 07:29:33.321402 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:33.321411 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:33.321439 (+ 28us) server_negotiation.cc:407] Connection header received
0925 07:29:33.321888 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:33.321893 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:33.321936 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:33.322072 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:33.322672 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:33.323562 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:33.324387 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:33.324772 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:33.324881 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:33.327457 (+ 2576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:33.327467 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:33.327468 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:29:33.327477 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:33.329520 (+ 2043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:33.329950 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:33.329954 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:33.329954 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:33.330055 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:33.330298 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:33.330299 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:33.330300 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:33.330575 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:33.330730 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:33.330883 (+ 153us) server_negotiation.cc:299] Negotiation successful
0925 07:29:33.330977 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":90,"threads_started":1}
I20260925 07:29:33.331843 27056 catalog_manager.cc:1347] Generated new cluster ID: 570eca837d8c45bfacdb0ec028256882
I20260925 07:29:33.331899 27056 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 07:29:33.332051 5753 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 07:29:33.339432 27045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:33.333325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.72:35043 (local address 127.0.0.1:37225)
0925 07:29:33.333359 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:33.333361 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:33.333460 (+ 99us) server_negotiation.cc:407] Connection header received
0925 07:29:33.333514 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:33.333515 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:33.333524 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:33.333577 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:33.334075 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:33.334535 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:33.335225 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:33.335471 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:33.335557 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:33.336635 (+ 1078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:33.336637 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:33.336637 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:29:33.336643 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:33.338059 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:33.338405 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:33.338406 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:33.338406 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:33.338454 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:33.338833 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:33.338834 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:33.338835 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:33.338957 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:33.339074 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:33.339245 (+ 171us) server_negotiation.cc:299] Negotiation successful
0925 07:29:33.339313 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 07:29:33.347568 27056 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 07:29:33.348024 27056 catalog_manager.cc:1504] Loading token signing keys...
I20260925 07:29:33.355156 27056 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e841e9bd927a422eac3b3d6e359b4c0a: Generated new TSK 0
I20260925 07:29:33.355324 27056 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 07:29:33.358487 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 26974
W20260925 07:29:33.362926 26959 connection.cc:535] client connection to 127.0.0.1:37225 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37225): Connection reset by peer (error 104)
2026-09-25T07:29:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1180 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790321110323796-5753-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 07:29:33 dist-test-slave-1drq krb5kdc[27073](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 07:29:33 dist-test-slave-1drq krb5kdc[27073](info): set up 2 sockets
Sep 25 07:29:33 dist-test-slave-1drq krb5kdc[27073](info): commencing operation
krb5kdc: starting...
W20260925 07:29:34.313064 5753 mini_kdc.cc:121] Time spent starting KDC: real 0.937s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Sep 25 07:29:34 dist-test-slave-1drq krb5kdc[27073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321374, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T07:29:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T07:29:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790321110323796-5753-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790321110323796-5753-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 07:29:34.415163 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37913
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790321110323796-5753-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37913
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:29:34.486577 27086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:29:34.486954 27086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:29:34.486996 27086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:29:34.488137 27086 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:29:34.488209 27086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:29:34.488232 27086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:29:34.488255 27086 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:29:34.488284 27086 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:29:34.489724 27086 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790321110323796-5753-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37913
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790321110323796-5753-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37913
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27086
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:29:34.489997 27086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:29:34.490232 27086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:29:34.494868 27092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:34.494863 27091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:34.494927 27086 server_base.cc:1034] running on GCE node
W20260925 07:29:34.494863 27094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:34.495280 27086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:29:34.495558 27086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:29:34.496773 27086 hybrid_clock.cc:648] HybridClock initialized: now 1790321374496759 us; error 22 us; skew 500 ppm
Sep 25 07:29:34 dist-test-slave-1drq krb5kdc[27073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321374, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:29:34.505682 27086 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 07:29:34.506130 27086 webserver.cc:460] Webserver started at http://127.0.0.1:40715/ using document root <none> and password file <none>
I20260925 07:29:34.506348 27086 fs_manager.cc:359] Metadata directory not provided
I20260925 07:29:34.506413 27086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:29:34.506582 27086 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:29:34.507607 27086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790321110323796-5753-0/minicluster-data/master-0/data/instance:
uuid: "d17e5deb572e4591a3e783430a7f66b8"
format_stamp: "Formatted at 2026-09-25 07:29:34 on dist-test-slave-1drq"
server_key: "1d0504af58ac63f6d01df674389086bf"
server_key_iv: "e4e82a0d43e7dc4e02ad372168ffbe9a"
server_key_version: "encryptionkey@0"
I20260925 07:29:34.507970 27086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790321110323796-5753-0/minicluster-data/master-0/wal/instance:
uuid: "d17e5deb572e4591a3e783430a7f66b8"
format_stamp: "Formatted at 2026-09-25 07:29:34 on dist-test-slave-1drq"
server_key: "1d0504af58ac63f6d01df674389086bf"
server_key_iv: "e4e82a0d43e7dc4e02ad372168ffbe9a"
server_key_version: "encryptionkey@0"
I20260925 07:29:34.509212 27086 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260925 07:29:34.509867 27101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:34.510020 27086 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:34.510121 27086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "d17e5deb572e4591a3e783430a7f66b8"
format_stamp: "Formatted at 2026-09-25 07:29:34 on dist-test-slave-1drq"
server_key: "1d0504af58ac63f6d01df674389086bf"
server_key_iv: "e4e82a0d43e7dc4e02ad372168ffbe9a"
server_key_version: "encryptionkey@0"
I20260925 07:29:34.510211 27086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:29:34.523017 27086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:29:34.524739 27086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:29:34.524894 27086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:29:34.528934 27086 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37913
I20260925 07:29:34.528959 27153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37913 every 8 connection(s)
I20260925 07:29:34.529362 27086 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:29:34.530112 27154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:29:34.532140 27154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d17e5deb572e4591a3e783430a7f66b8: Bootstrap starting.
I20260925 07:29:34.532639 27154 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d17e5deb572e4591a3e783430a7f66b8: Neither blocks nor log segments found. Creating new log.
I20260925 07:29:34.532847 27154 log.cc:824] T 00000000000000000000000000000000 P d17e5deb572e4591a3e783430a7f66b8: Log is configured to *not* fsync() on all Append() calls
I20260925 07:29:34.533479 27154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d17e5deb572e4591a3e783430a7f66b8: No bootstrap required, opened a new log
I20260925 07:29:34.534745 27154 raft_consensus.cc:348] T 00000000000000000000000000000000 P d17e5deb572e4591a3e783430a7f66b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d17e5deb572e4591a3e783430a7f66b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37913 } }
I20260925 07:29:34.534863 27154 raft_consensus.cc:374] T 00000000000000000000000000000000 P d17e5deb572e4591a3e783430a7f66b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:29:34.534883 27154 raft_consensus.cc:729] T 00000000000000000000000000000000 P d17e5deb572e4591a3e783430a7f66b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d17e5deb572e4591a3e783430a7f66b8, State: Initialized, Role: FOLLOWER
I20260925 07:29:34.534993 27154 consensus_queue.cc:260] T 00000000000000000000000000000000 P d17e5deb572e4591a3e783430a7f66b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d17e5deb572e4591a3e783430a7f66b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37913 } }
I20260925 07:29:34.535063 27154 raft_consensus.cc:388] T 00000000000000000000000000000000 P d17e5deb572e4591a3e783430a7f66b8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 07:29:34.535081 27154 raft_consensus.cc:482] T 00000000000000000000000000000000 P d17e5deb572e4591a3e783430a7f66b8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 07:29:34.535105 27154 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d17e5deb572e4591a3e783430a7f66b8 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:29:34.535506 27154 raft_consensus.cc:504] T 00000000000000000000000000000000 P d17e5deb572e4591a3e783430a7f66b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d17e5deb572e4591a3e783430a7f66b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37913 } }
I20260925 07:29:34.535575 27154 leader_election.cc:302] T 00000000000000000000000000000000 P d17e5deb572e4591a3e783430a7f66b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d17e5deb572e4591a3e783430a7f66b8; no voters:
I20260925 07:29:34.535671 27154 leader_election.cc:288] T 00000000000000000000000000000000 P d17e5deb572e4591a3e783430a7f66b8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 07:29:34.535737 27157 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d17e5deb572e4591a3e783430a7f66b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:29:34.535878 27157 raft_consensus.cc:686] T 00000000000000000000000000000000 P d17e5deb572e4591a3e783430a7f66b8 [term 1 LEADER]: Becoming Leader. State: Replica: d17e5deb572e4591a3e783430a7f66b8, State: Running, Role: LEADER
I20260925 07:29:34.535912 27154 sys_catalog.cc:564] T 00000000000000000000000000000000 P d17e5deb572e4591a3e783430a7f66b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:29:34.535989 27157 consensus_queue.cc:237] T 00000000000000000000000000000000 P d17e5deb572e4591a3e783430a7f66b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d17e5deb572e4591a3e783430a7f66b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37913 } }
I20260925 07:29:34.536412 27159 sys_catalog.cc:455] T 00000000000000000000000000000000 P d17e5deb572e4591a3e783430a7f66b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d17e5deb572e4591a3e783430a7f66b8. Latest consensus state: current_term: 1 leader_uuid: "d17e5deb572e4591a3e783430a7f66b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d17e5deb572e4591a3e783430a7f66b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37913 } } }
I20260925 07:29:34.536513 27159 sys_catalog.cc:458] T 00000000000000000000000000000000 P d17e5deb572e4591a3e783430a7f66b8 [sys.catalog]: This master's current role is: LEADER
I20260925 07:29:34.536504 27158 sys_catalog.cc:455] T 00000000000000000000000000000000 P d17e5deb572e4591a3e783430a7f66b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d17e5deb572e4591a3e783430a7f66b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d17e5deb572e4591a3e783430a7f66b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37913 } } }
I20260925 07:29:34.536913 27158 sys_catalog.cc:458] T 00000000000000000000000000000000 P d17e5deb572e4591a3e783430a7f66b8 [sys.catalog]: This master's current role is: LEADER
I20260925 07:29:34.536949 27166 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 07:29:34.537475 27166 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 07:29:34.539024 27166 catalog_manager.cc:1347] Generated new cluster ID: 463cdda5c3ed44ddbdde9fbd88fafda6
I20260925 07:29:34.539065 27166 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 07:29:34.539338 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 27086
I20260925 07:29:34.539438 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790321110323796-5753-0/minicluster-data/master-0/wal/instance
I20260925 07:29:34.539642 5753 external_mini_cluster.cc:1428] Setting key 372f2e85728649dcfa37dc5e12baac95
I20260925 07:29:34.545323 27177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:34.540223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.72:44211 (local address 127.0.0.1:37913)
0925 07:29:34.540374 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:34.540381 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:34.540408 (+ 27us) server_negotiation.cc:407] Connection header received
0925 07:29:34.540516 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:34.540523 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:34.540572 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:34.540707 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:34.540937 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:34.542320 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:34.543309 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:34.543847 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:34.543957 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:34.544595 (+ 638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:34.544604 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:34.544605 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:29:34.544791 (+ 186us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:34.544825 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:29:34.544837 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:34.544925 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:34.545019 (+ 94us) server_negotiation.cc:299] Negotiation successful
0925 07:29:34.545150 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":56,"threads_started":1}
I20260925 07:29:34.546151 5753 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 07:29:34.549132 27166 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 07:29:34.549544 27166 catalog_manager.cc:1504] Loading token signing keys...
I20260925 07:29:34.556278 27177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:34.547441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.72:48789 (local address 127.0.0.1:37913)
0925 07:29:34.547469 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:34.547471 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:34.553281 (+ 5810us) server_negotiation.cc:407] Connection header received
0925 07:29:34.553331 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:34.553332 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:34.553341 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:34.553395 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:34.553516 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:34.554153 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:34.555112 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:34.555413 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:34.555476 (+ 63us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:34.556070 (+ 594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:34.556071 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:34.556072 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:29:34.556075 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:34.556086 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:29:34.556090 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:34.556143 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:34.556210 (+ 67us) server_negotiation.cc:299] Negotiation successful
0925 07:29:34.556221 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260925 07:29:34.557381 27166 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d17e5deb572e4591a3e783430a7f66b8: Generated new TSK 0
I20260925 07:29:34.557516 27166 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 07:29:34.562661 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 27086
W20260925 07:29:34.566192 27071 connection.cc:535] client connection to 127.0.0.1:37913 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37913): Connection reset by peer (error 104)
2026-09-25T07:29:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1203 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790321110323796-5753-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 07:29:34 dist-test-slave-1drq krb5kdc[27185](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 07:29:34 dist-test-slave-1drq krb5kdc[27185](info): set up 2 sockets
Sep 25 07:29:34 dist-test-slave-1drq krb5kdc[27185](info): commencing operation
krb5kdc: starting...
W20260925 07:29:35.500876 5753 mini_kdc.cc:121] Time spent starting KDC: real 0.922s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Sep 25 07:29:35 dist-test-slave-1drq krb5kdc[27185](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321375, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T07:29:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T07:29:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790321110323796-5753-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790321110323796-5753-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 07:29:35.603758 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40241
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790321110323796-5753-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40241
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:29:35.677212 27198 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:29:35.677364 27198 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:29:35.677385 27198 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:29:35.678361 27198 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:29:35.678404 27198 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:29:35.678418 27198 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:29:35.678467 27198 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:29:35.678493 27198 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:29:35.679842 27198 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790321110323796-5753-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40241
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790321110323796-5753-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27198
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:29:35.680073 27198 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:29:35.680281 27198 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:29:35.684540 27204 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:35.684686 27198 server_base.cc:1034] running on GCE node
W20260925 07:29:35.684556 27206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:35.684885 27203 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:35.685173 27198 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:29:35.685393 27198 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:29:35.686553 27198 hybrid_clock.cc:648] HybridClock initialized: now 1790321375686542 us; error 23 us; skew 500 ppm
Sep 25 07:29:35 dist-test-slave-1drq krb5kdc[27185](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321375, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:29:35.688699 27198 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 07:29:35.689136 27198 webserver.cc:460] Webserver started at http://127.0.0.1:40261/ using document root <none> and password file <none>
I20260925 07:29:35.689342 27198 fs_manager.cc:359] Metadata directory not provided
I20260925 07:29:35.689399 27198 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:29:35.689517 27198 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:29:35.690330 27198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790321110323796-5753-0/minicluster-data/master-0/data/instance:
uuid: "25892b3b1fd747febc4de9a4ac64708d"
format_stamp: "Formatted at 2026-09-25 07:29:35 on dist-test-slave-1drq"
server_key: "c28bb19c35c047b2b4b8ddff29e982e1"
server_key_iv: "4c4c3f008eb27bc00316e85d293065c8"
server_key_version: "encryptionkey@0"
I20260925 07:29:35.690657 27198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790321110323796-5753-0/minicluster-data/master-0/wal/instance:
uuid: "25892b3b1fd747febc4de9a4ac64708d"
format_stamp: "Formatted at 2026-09-25 07:29:35 on dist-test-slave-1drq"
server_key: "c28bb19c35c047b2b4b8ddff29e982e1"
server_key_iv: "4c4c3f008eb27bc00316e85d293065c8"
server_key_version: "encryptionkey@0"
I20260925 07:29:35.691888 27198 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 07:29:35.692524 27213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:35.692688 27198 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:35.692766 27198 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "25892b3b1fd747febc4de9a4ac64708d"
format_stamp: "Formatted at 2026-09-25 07:29:35 on dist-test-slave-1drq"
server_key: "c28bb19c35c047b2b4b8ddff29e982e1"
server_key_iv: "4c4c3f008eb27bc00316e85d293065c8"
server_key_version: "encryptionkey@0"
I20260925 07:29:35.692842 27198 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:29:35.695756 27198 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:29:35.697391 27198 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:29:35.697520 27198 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:29:35.701122 27198 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40241
I20260925 07:29:35.701151 27265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40241 every 8 connection(s)
I20260925 07:29:35.701443 27198 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:29:35.702286 27266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:29:35.704077 27266 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25892b3b1fd747febc4de9a4ac64708d: Bootstrap starting.
I20260925 07:29:35.704571 27266 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 25892b3b1fd747febc4de9a4ac64708d: Neither blocks nor log segments found. Creating new log.
I20260925 07:29:35.704830 27266 log.cc:824] T 00000000000000000000000000000000 P 25892b3b1fd747febc4de9a4ac64708d: Log is configured to *not* fsync() on all Append() calls
I20260925 07:29:35.705567 27266 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25892b3b1fd747febc4de9a4ac64708d: No bootstrap required, opened a new log
I20260925 07:29:35.706974 27266 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25892b3b1fd747febc4de9a4ac64708d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25892b3b1fd747febc4de9a4ac64708d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40241 } }
I20260925 07:29:35.707086 27266 raft_consensus.cc:374] T 00000000000000000000000000000000 P 25892b3b1fd747febc4de9a4ac64708d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:29:35.707106 27266 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25892b3b1fd747febc4de9a4ac64708d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25892b3b1fd747febc4de9a4ac64708d, State: Initialized, Role: FOLLOWER
I20260925 07:29:35.707260 27266 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25892b3b1fd747febc4de9a4ac64708d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25892b3b1fd747febc4de9a4ac64708d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40241 } }
I20260925 07:29:35.707340 27266 raft_consensus.cc:388] T 00000000000000000000000000000000 P 25892b3b1fd747febc4de9a4ac64708d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 07:29:35.707377 27266 raft_consensus.cc:482] T 00000000000000000000000000000000 P 25892b3b1fd747febc4de9a4ac64708d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 07:29:35.707423 27266 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25892b3b1fd747febc4de9a4ac64708d [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:29:35.707448 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 27198
I20260925 07:29:35.707553 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790321110323796-5753-0/minicluster-data/master-0/wal/instance
I20260925 07:29:35.707737 5753 external_mini_cluster.cc:1428] Setting key e8a19bb61fea6d989e92f7d503c3a8cb
I20260925 07:29:35.708037 27266 raft_consensus.cc:504] T 00000000000000000000000000000000 P 25892b3b1fd747febc4de9a4ac64708d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25892b3b1fd747febc4de9a4ac64708d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40241 } }
I20260925 07:29:35.708142 27266 leader_election.cc:302] T 00000000000000000000000000000000 P 25892b3b1fd747febc4de9a4ac64708d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 25892b3b1fd747febc4de9a4ac64708d; no voters:
I20260925 07:29:35.708277 27266 leader_election.cc:288] T 00000000000000000000000000000000 P 25892b3b1fd747febc4de9a4ac64708d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 07:29:35.708303 27271 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 25892b3b1fd747febc4de9a4ac64708d [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:29:35.708439 27271 raft_consensus.cc:686] T 00000000000000000000000000000000 P 25892b3b1fd747febc4de9a4ac64708d [term 1 LEADER]: Becoming Leader. State: Replica: 25892b3b1fd747febc4de9a4ac64708d, State: Running, Role: LEADER
I20260925 07:29:35.708505 27266 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25892b3b1fd747febc4de9a4ac64708d [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:29:35.708546 27271 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25892b3b1fd747febc4de9a4ac64708d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25892b3b1fd747febc4de9a4ac64708d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40241 } }
I20260925 07:29:35.709213 27272 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25892b3b1fd747febc4de9a4ac64708d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "25892b3b1fd747febc4de9a4ac64708d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25892b3b1fd747febc4de9a4ac64708d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40241 } } }
I20260925 07:29:35.709296 27272 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25892b3b1fd747febc4de9a4ac64708d [sys.catalog]: This master's current role is: LEADER
I20260925 07:29:35.709604 27273 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25892b3b1fd747febc4de9a4ac64708d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25892b3b1fd747febc4de9a4ac64708d. Latest consensus state: current_term: 1 leader_uuid: "25892b3b1fd747febc4de9a4ac64708d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25892b3b1fd747febc4de9a4ac64708d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40241 } } }
I20260925 07:29:35.709697 27273 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25892b3b1fd747febc4de9a4ac64708d [sys.catalog]: This master's current role is: LEADER
I20260925 07:29:35.710049 27280 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 07:29:35.710759 27280 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 07:29:35.711297 27270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:35.708099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.72:37353 (local address 127.0.0.1:40241)
0925 07:29:35.708258 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:35.708268 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:35.709367 (+ 1099us) server_negotiation.cc:407] Connection header received
0925 07:29:35.709626 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:35.709633 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:35.709665 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:35.709786 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:35.710860 (+ 1074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:35.710864 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:35.710864 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:29:35.710869 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:35.710913 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:29:35.710927 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:35.711003 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:35.711080 (+ 77us) server_negotiation.cc:299] Negotiation successful
0925 07:29:35.711119 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":82,"threads_started":1}
I20260925 07:29:35.712215 5753 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 07:29:35.712302 27280 catalog_manager.cc:1347] Generated new cluster ID: 6257b82310ec4e26b0d38c62fc0e33f2
I20260925 07:29:35.712352 27280 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 07:29:35.714636 27270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:35.713541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.72:48659 (local address 127.0.0.1:40241)
0925 07:29:35.713595 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:35.713597 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:35.713749 (+ 152us) server_negotiation.cc:407] Connection header received
0925 07:29:35.713838 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:35.713839 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:35.713843 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:35.713893 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:35.714436 (+ 543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:35.714437 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:35.714437 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:29:35.714440 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:35.714452 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:29:35.714456 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:35.714503 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:35.714560 (+ 57us) server_negotiation.cc:299] Negotiation successful
0925 07:29:35.714574 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260925 07:29:35.718865 27280 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 07:29:35.718992 27280 catalog_manager.cc:1504] Loading token signing keys...
I20260925 07:29:35.724507 27280 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 25892b3b1fd747febc4de9a4ac64708d: Generated new TSK 0
I20260925 07:29:35.724653 27280 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 07:29:35.732496 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 27198
2026-09-25T07:29:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1170 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790321110323796-5753-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 07:29:35 dist-test-slave-1drq krb5kdc[27297](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 07:29:35 dist-test-slave-1drq krb5kdc[27297](info): set up 2 sockets
Sep 25 07:29:35 dist-test-slave-1drq krb5kdc[27297](info): commencing operation
krb5kdc: starting...
W20260925 07:29:36.684727 5753 mini_kdc.cc:121] Time spent starting KDC: real 0.935s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Sep 25 07:29:36 dist-test-slave-1drq krb5kdc[27297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321376, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T07:29:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T07:29:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
I20260925 07:29:36.787098 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:34149
--webserver_interface=127.5.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:46627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.158.126:34149
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:29:36.856251 27310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:29:36.856448 27310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:29:36.856484 27310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:29:36.857507 27310 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:29:36.857563 27310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:29:36.857584 27310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:29:36.857607 27310 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:29:36.857625 27310 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:29:36.858844 27310 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:46627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790321110323796-5753-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.158.126:34149
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:34149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27310
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:29:36.859086 27310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:29:36.859287 27310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:29:36.863724 27315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:36.863837 27310 server_base.cc:1034] running on GCE node
W20260925 07:29:36.863795 27316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:36.863916 27318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:36.864112 27310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:29:36.864300 27310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:29:36.865419 27310 hybrid_clock.cc:648] HybridClock initialized: now 1790321376865407 us; error 26 us; skew 500 ppm
Sep 25 07:29:36 dist-test-slave-1drq krb5kdc[27297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321376, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:29:36.867436 27310 init.cc:377] Logged in from keytab as kudu/127.5.158.126@KRBTEST.COM (short username kudu)
I20260925 07:29:36.867823 27310 webserver.cc:460] Webserver started at http://127.5.158.126:36009/ using document root <none> and password file <none>
I20260925 07:29:36.868016 27310 fs_manager.cc:359] Metadata directory not provided
I20260925 07:29:36.868052 27310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:29:36.868170 27310 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:29:36.869014 27310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790321110323796-5753-0/minicluster-data/master-0/data/instance:
uuid: "9663f4c568eb4575940bc2e6f298aeda"
format_stamp: "Formatted at 2026-09-25 07:29:36 on dist-test-slave-1drq"
server_key: "c204112a890473f9ceeadffc51ce9021"
server_key_iv: "65778eb3992438d42ec6809066287992"
server_key_version: "encryptionkey@0"
I20260925 07:29:36.869352 27310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790321110323796-5753-0/minicluster-data/master-0/wal/instance:
uuid: "9663f4c568eb4575940bc2e6f298aeda"
format_stamp: "Formatted at 2026-09-25 07:29:36 on dist-test-slave-1drq"
server_key: "c204112a890473f9ceeadffc51ce9021"
server_key_iv: "65778eb3992438d42ec6809066287992"
server_key_version: "encryptionkey@0"
I20260925 07:29:36.870433 27310 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 07:29:36.871153 27325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:36.871335 27310 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 07:29:36.871404 27310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "9663f4c568eb4575940bc2e6f298aeda"
format_stamp: "Formatted at 2026-09-25 07:29:36 on dist-test-slave-1drq"
server_key: "c204112a890473f9ceeadffc51ce9021"
server_key_iv: "65778eb3992438d42ec6809066287992"
server_key_version: "encryptionkey@0"
I20260925 07:29:36.871487 27310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:29:36.914878 27310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:29:36.916814 27310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:29:36.916914 27310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:29:36.920326 27310 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.126:34149
I20260925 07:29:36.920382 27377 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.126:34149 every 8 connection(s)
I20260925 07:29:36.920715 27310 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:29:36.920907 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 27310
I20260925 07:29:36.921048 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790321110323796-5753-0/minicluster-data/master-0/wal/instance
I20260925 07:29:36.921334 5753 external_mini_cluster.cc:1428] Setting key e82e3b00a32e59d3e4c0f5d67be4ba0b
I20260925 07:29:36.921806 27378 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:29:36.924690 27378 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9663f4c568eb4575940bc2e6f298aeda: Bootstrap starting.
I20260925 07:29:36.925344 27378 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9663f4c568eb4575940bc2e6f298aeda: Neither blocks nor log segments found. Creating new log.
I20260925 07:29:36.925674 27378 log.cc:824] T 00000000000000000000000000000000 P 9663f4c568eb4575940bc2e6f298aeda: Log is configured to *not* fsync() on all Append() calls
I20260925 07:29:36.926359 27378 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9663f4c568eb4575940bc2e6f298aeda: No bootstrap required, opened a new log
Sep 25 07:29:36 dist-test-slave-1drq krb5kdc[27297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321376, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:29:36.928552 27378 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9663f4c568eb4575940bc2e6f298aeda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9663f4c568eb4575940bc2e6f298aeda" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 34149 } }
I20260925 07:29:36.928716 27378 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9663f4c568eb4575940bc2e6f298aeda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:29:36.928758 27378 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9663f4c568eb4575940bc2e6f298aeda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9663f4c568eb4575940bc2e6f298aeda, State: Initialized, Role: FOLLOWER
I20260925 07:29:36.928916 27378 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9663f4c568eb4575940bc2e6f298aeda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9663f4c568eb4575940bc2e6f298aeda" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 34149 } }
I20260925 07:29:36.928992 27378 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9663f4c568eb4575940bc2e6f298aeda [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 07:29:36.929013 27378 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9663f4c568eb4575940bc2e6f298aeda [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 07:29:36.929068 27378 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9663f4c568eb4575940bc2e6f298aeda [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:29:36.929598 27378 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9663f4c568eb4575940bc2e6f298aeda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9663f4c568eb4575940bc2e6f298aeda" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 34149 } }
I20260925 07:29:36.929735 27378 leader_election.cc:302] T 00000000000000000000000000000000 P 9663f4c568eb4575940bc2e6f298aeda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9663f4c568eb4575940bc2e6f298aeda; no voters:
I20260925 07:29:36.929885 27378 leader_election.cc:288] T 00000000000000000000000000000000 P 9663f4c568eb4575940bc2e6f298aeda [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 07:29:36.929952 27383 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9663f4c568eb4575940bc2e6f298aeda [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:29:36.930079 27383 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9663f4c568eb4575940bc2e6f298aeda [term 1 LEADER]: Becoming Leader. State: Replica: 9663f4c568eb4575940bc2e6f298aeda, State: Running, Role: LEADER
I20260925 07:29:36.930128 27378 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9663f4c568eb4575940bc2e6f298aeda [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:29:36.930152 27383 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9663f4c568eb4575940bc2e6f298aeda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9663f4c568eb4575940bc2e6f298aeda" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 34149 } }
I20260925 07:29:36.930441 27384 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9663f4c568eb4575940bc2e6f298aeda [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9663f4c568eb4575940bc2e6f298aeda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9663f4c568eb4575940bc2e6f298aeda" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 34149 } } }
I20260925 07:29:36.930505 27384 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9663f4c568eb4575940bc2e6f298aeda [sys.catalog]: This master's current role is: LEADER
I20260925 07:29:36.930704 27385 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9663f4c568eb4575940bc2e6f298aeda [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9663f4c568eb4575940bc2e6f298aeda. Latest consensus state: current_term: 1 leader_uuid: "9663f4c568eb4575940bc2e6f298aeda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9663f4c568eb4575940bc2e6f298aeda" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 34149 } } }
I20260925 07:29:36.931030 27385 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9663f4c568eb4575940bc2e6f298aeda [sys.catalog]: This master's current role is: LEADER
I20260925 07:29:36.931222 27388 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 07:29:36.931607 27381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:36.921823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49691 (local address 127.5.158.126:34149)
0925 07:29:36.922012 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:36.922020 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:36.922044 (+ 24us) server_negotiation.cc:407] Connection header received
0925 07:29:36.922444 (+ 400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:36.922449 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:36.922488 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:36.922587 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:36.923301 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:36.924101 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:36.925030 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:36.925487 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:36.925556 (+ 69us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:36.927930 (+ 2374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:36.927934 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:36.927934 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:29:36.927942 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:36.929565 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:36.930010 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:36.930013 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:36.930013 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:36.930101 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:36.930342 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:36.930343 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:36.930343 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:36.930623 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:36.930801 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:36.931280 (+ 479us) server_negotiation.cc:299] Negotiation successful
0925 07:29:36.931391 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":93,"threads_started":1}
I20260925 07:29:36.931710 27388 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 07:29:36.932396 5753 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 07:29:36.933212 27388 catalog_manager.cc:1347] Generated new cluster ID: cfacfeb0cbc7498dbe0038ce79ec9128
I20260925 07:29:36.933256 27388 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 07:29:36.937857 27388 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 07:29:36.938480 27388 catalog_manager.cc:1504] Loading token signing keys...
I20260925 07:29:36.940064 27381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:36.933822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47573 (local address 127.5.158.126:34149)
0925 07:29:36.933850 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:36.933852 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:36.933953 (+ 101us) server_negotiation.cc:407] Connection header received
0925 07:29:36.934001 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:36.934002 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:36.934009 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:36.934059 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:36.934596 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:36.935221 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:36.936108 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:36.936396 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:36.936449 (+ 53us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:36.937713 (+ 1264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:36.937714 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:36.937714 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:29:36.937720 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:36.939022 (+ 1302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:36.939358 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:36.939358 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:36.939359 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:36.939410 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:36.939605 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:36.939606 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:36.939607 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:36.939702 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:36.939795 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:36.939928 (+ 133us) server_negotiation.cc:299] Negotiation successful
0925 07:29:36.939983 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260925 07:29:36.946286 27388 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9663f4c568eb4575940bc2e6f298aeda: Generated new TSK 0
I20260925 07:29:36.946415 27388 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 07:29:36.950690 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 27310
W20260925 07:29:36.954587 27295 connection.cc:535] client connection to 127.5.158.126:34149 recv error: Network error: failed to read from TLS socket (remote: 127.5.158.126:34149): Connection reset by peer (error 104)
2026-09-25T07:29:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1218 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790321110323796-5753-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 07:29:36 dist-test-slave-1drq krb5kdc[27409](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 07:29:37 dist-test-slave-1drq krb5kdc[27409](info): set up 2 sockets
Sep 25 07:29:37 dist-test-slave-1drq krb5kdc[27409](info): commencing operation
krb5kdc: starting...
W20260925 07:29:37.892951 5753 mini_kdc.cc:121] Time spent starting KDC: real 0.925s user 0.001s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Sep 25 07:29:37 dist-test-slave-1drq krb5kdc[27409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321377, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T07:29:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T07:29:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
I20260925 07:29:37.997239 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:34035
--webserver_interface=127.5.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:41503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.158.126:34035
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:29:38.068042 27422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:29:38.068250 27422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:29:38.068284 27422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:29:38.069278 27422 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:29:38.069334 27422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:29:38.069353 27422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:29:38.069371 27422 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:29:38.069398 27422 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:29:38.070609 27422 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:41503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790321110323796-5753-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.158.126:34035
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:34035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27422
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:29:38.070833 27422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:29:38.071018 27422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:29:38.075582 27428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:38.075621 27430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:38.075733 27427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:38.075780 27422 server_base.cc:1034] running on GCE node
I20260925 07:29:38.075986 27422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:29:38.076232 27422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:29:38.077378 27422 hybrid_clock.cc:648] HybridClock initialized: now 1790321378077367 us; error 28 us; skew 500 ppm
Sep 25 07:29:38 dist-test-slave-1drq krb5kdc[27409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321378, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:29:38.079627 27422 init.cc:377] Logged in from keytab as kudu/127.5.158.126@KRBTEST.COM (short username kudu)
I20260925 07:29:38.080022 27422 webserver.cc:460] Webserver started at http://127.5.158.126:46211/ using document root <none> and password file <none>
I20260925 07:29:38.080237 27422 fs_manager.cc:359] Metadata directory not provided
I20260925 07:29:38.080299 27422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:29:38.080466 27422 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:29:38.081471 27422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790321110323796-5753-0/minicluster-data/master-0/data/instance:
uuid: "564ada6833c54fbfa1fe21c7fcc548ae"
format_stamp: "Formatted at 2026-09-25 07:29:38 on dist-test-slave-1drq"
server_key: "a5d5202034bdbada37351ff4f8bd80b4"
server_key_iv: "db35fb833394752164c79158088f84fa"
server_key_version: "encryptionkey@0"
I20260925 07:29:38.081804 27422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790321110323796-5753-0/minicluster-data/master-0/wal/instance:
uuid: "564ada6833c54fbfa1fe21c7fcc548ae"
format_stamp: "Formatted at 2026-09-25 07:29:38 on dist-test-slave-1drq"
server_key: "a5d5202034bdbada37351ff4f8bd80b4"
server_key_iv: "db35fb833394752164c79158088f84fa"
server_key_version: "encryptionkey@0"
I20260925 07:29:38.083022 27422 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 07:29:38.083662 27437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:38.083806 27422 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:29:38.083904 27422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "564ada6833c54fbfa1fe21c7fcc548ae"
format_stamp: "Formatted at 2026-09-25 07:29:38 on dist-test-slave-1drq"
server_key: "a5d5202034bdbada37351ff4f8bd80b4"
server_key_iv: "db35fb833394752164c79158088f84fa"
server_key_version: "encryptionkey@0"
I20260925 07:29:38.083971 27422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:29:38.095507 27422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:29:38.097069 27422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:29:38.097277 27422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:29:38.100503 27422 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.126:34035
I20260925 07:29:38.100531 27489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.126:34035 every 8 connection(s)
I20260925 07:29:38.100843 27422 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:29:38.101655 27490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:29:38.103600 27490 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 564ada6833c54fbfa1fe21c7fcc548ae: Bootstrap starting.
I20260925 07:29:38.104072 27490 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 564ada6833c54fbfa1fe21c7fcc548ae: Neither blocks nor log segments found. Creating new log.
I20260925 07:29:38.104272 27490 log.cc:824] T 00000000000000000000000000000000 P 564ada6833c54fbfa1fe21c7fcc548ae: Log is configured to *not* fsync() on all Append() calls
I20260925 07:29:38.104811 27490 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 564ada6833c54fbfa1fe21c7fcc548ae: No bootstrap required, opened a new log
I20260925 07:29:38.106242 27490 raft_consensus.cc:348] T 00000000000000000000000000000000 P 564ada6833c54fbfa1fe21c7fcc548ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "564ada6833c54fbfa1fe21c7fcc548ae" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 34035 } }
I20260925 07:29:38.106359 27490 raft_consensus.cc:374] T 00000000000000000000000000000000 P 564ada6833c54fbfa1fe21c7fcc548ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:29:38.106380 27490 raft_consensus.cc:729] T 00000000000000000000000000000000 P 564ada6833c54fbfa1fe21c7fcc548ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 564ada6833c54fbfa1fe21c7fcc548ae, State: Initialized, Role: FOLLOWER
I20260925 07:29:38.106500 27490 consensus_queue.cc:260] T 00000000000000000000000000000000 P 564ada6833c54fbfa1fe21c7fcc548ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "564ada6833c54fbfa1fe21c7fcc548ae" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 34035 } }
I20260925 07:29:38.106581 27490 raft_consensus.cc:388] T 00000000000000000000000000000000 P 564ada6833c54fbfa1fe21c7fcc548ae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 07:29:38.106603 27490 raft_consensus.cc:482] T 00000000000000000000000000000000 P 564ada6833c54fbfa1fe21c7fcc548ae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 07:29:38.106630 27490 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 564ada6833c54fbfa1fe21c7fcc548ae [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:29:38.107347 27490 raft_consensus.cc:504] T 00000000000000000000000000000000 P 564ada6833c54fbfa1fe21c7fcc548ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "564ada6833c54fbfa1fe21c7fcc548ae" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 34035 } }
I20260925 07:29:38.107422 27490 leader_election.cc:302] T 00000000000000000000000000000000 P 564ada6833c54fbfa1fe21c7fcc548ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 564ada6833c54fbfa1fe21c7fcc548ae; no voters:
I20260925 07:29:38.107524 27490 leader_election.cc:288] T 00000000000000000000000000000000 P 564ada6833c54fbfa1fe21c7fcc548ae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 07:29:38.107584 27493 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 564ada6833c54fbfa1fe21c7fcc548ae [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:29:38.107743 27493 raft_consensus.cc:686] T 00000000000000000000000000000000 P 564ada6833c54fbfa1fe21c7fcc548ae [term 1 LEADER]: Becoming Leader. State: Replica: 564ada6833c54fbfa1fe21c7fcc548ae, State: Running, Role: LEADER
I20260925 07:29:38.107774 27490 sys_catalog.cc:564] T 00000000000000000000000000000000 P 564ada6833c54fbfa1fe21c7fcc548ae [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:29:38.107843 27493 consensus_queue.cc:237] T 00000000000000000000000000000000 P 564ada6833c54fbfa1fe21c7fcc548ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "564ada6833c54fbfa1fe21c7fcc548ae" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 34035 } }
I20260925 07:29:38.108207 27494 sys_catalog.cc:455] T 00000000000000000000000000000000 P 564ada6833c54fbfa1fe21c7fcc548ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "564ada6833c54fbfa1fe21c7fcc548ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "564ada6833c54fbfa1fe21c7fcc548ae" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 34035 } } }
I20260925 07:29:38.108224 27495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 564ada6833c54fbfa1fe21c7fcc548ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 564ada6833c54fbfa1fe21c7fcc548ae. Latest consensus state: current_term: 1 leader_uuid: "564ada6833c54fbfa1fe21c7fcc548ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "564ada6833c54fbfa1fe21c7fcc548ae" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 34035 } } }
I20260925 07:29:38.108311 27494 sys_catalog.cc:458] T 00000000000000000000000000000000 P 564ada6833c54fbfa1fe21c7fcc548ae [sys.catalog]: This master's current role is: LEADER
I20260925 07:29:38.108315 27495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 564ada6833c54fbfa1fe21c7fcc548ae [sys.catalog]: This master's current role is: LEADER
I20260925 07:29:38.108587 27502 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 07:29:38.109066 27502 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 07:29:38.110435 27502 catalog_manager.cc:1347] Generated new cluster ID: ca78fbf2178b4c06b5493d44f2e36902
I20260925 07:29:38.110481 27502 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 07:29:38.110566 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 27422
I20260925 07:29:38.110678 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790321110323796-5753-0/minicluster-data/master-0/wal/instance
I20260925 07:29:38.110877 5753 external_mini_cluster.cc:1428] Setting key 8fff0a0a1e9790f01d1f35ded297aa9e
I20260925 07:29:38.116358 27513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:38.111425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57019 (local address 127.5.158.126:34035)
0925 07:29:38.111576 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:38.111584 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:38.111612 (+ 28us) server_negotiation.cc:407] Connection header received
0925 07:29:38.111720 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:38.111726 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:38.111781 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:38.111899 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:38.112135 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:38.113381 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:38.114398 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:38.114910 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:38.115010 (+ 100us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:38.115729 (+ 719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:38.115732 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:38.115733 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:29:38.115929 (+ 196us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:38.115962 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:29:38.115974 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:38.116044 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:38.116174 (+ 130us) server_negotiation.cc:299] Negotiation successful
0925 07:29:38.116196 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":71,"threads_started":1}
I20260925 07:29:38.117238 5753 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 07:29:38.121786 27513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:38.118569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46785 (local address 127.5.158.126:34035)
0925 07:29:38.118600 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:38.118601 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:38.118713 (+ 112us) server_negotiation.cc:407] Connection header received
0925 07:29:38.118766 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:38.118766 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:38.118785 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:38.118831 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:38.118953 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:38.119589 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:38.120569 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:38.120888 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:38.120946 (+ 58us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:38.121550 (+ 604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:38.121551 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:38.121552 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:29:38.121555 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:38.121567 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:29:38.121571 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:38.121622 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:38.121710 (+ 88us) server_negotiation.cc:299] Negotiation successful
0925 07:29:38.121720 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260925 07:29:38.128788 27502 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 07:29:38.129304 27502 catalog_manager.cc:1504] Loading token signing keys...
I20260925 07:29:38.134193 27502 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 564ada6833c54fbfa1fe21c7fcc548ae: Generated new TSK 0
I20260925 07:29:38.134347 27502 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 07:29:38.135673 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 27422
W20260925 07:29:38.139386 27407 connection.cc:535] client connection to 127.5.158.126:34035 recv error: Network error: failed to read from TLS socket (remote: 127.5.158.126:34035): Connection reset by peer (error 104)
2026-09-25T07:29:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1184 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790321110323796-5753-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 07:29:38 dist-test-slave-1drq krb5kdc[27521](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 07:29:38 dist-test-slave-1drq krb5kdc[27521](info): set up 2 sockets
Sep 25 07:29:38 dist-test-slave-1drq krb5kdc[27521](info): commencing operation
krb5kdc: starting...
W20260925 07:29:39.077414 5753 mini_kdc.cc:121] Time spent starting KDC: real 0.925s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Sep 25 07:29:39 dist-test-slave-1drq krb5kdc[27521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321379, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T07:29:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T07:29:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
I20260925 07:29:39.182159 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:36213
--webserver_interface=127.5.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:34637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.158.126:36213
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:29:39.253558 27534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:29:39.253746 27534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:29:39.253779 27534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:29:39.254982 27534 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:29:39.255066 27534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:29:39.255100 27534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:29:39.255120 27534 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:29:39.255169 27534 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:29:39.257316 27534 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:34637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790321110323796-5753-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.158.126:36213
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:36213
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27534
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:29:39.257577 27534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:29:39.257794 27534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:29:39.262603 27539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:39.262699 27540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:39.262869 27542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:39.264072 27534 server_base.cc:1034] running on GCE node
I20260925 07:29:39.264256 27534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:29:39.264448 27534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:29:39.265568 27534 hybrid_clock.cc:648] HybridClock initialized: now 1790321379265556 us; error 37 us; skew 500 ppm
Sep 25 07:29:39 dist-test-slave-1drq krb5kdc[27521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321379, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:29:39.267767 27534 init.cc:377] Logged in from keytab as kudu/127.5.158.126@KRBTEST.COM (short username kudu)
I20260925 07:29:39.268157 27534 webserver.cc:460] Webserver started at http://127.5.158.126:35679/ using document root <none> and password file <none>
I20260925 07:29:39.268308 27534 fs_manager.cc:359] Metadata directory not provided
I20260925 07:29:39.268342 27534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:29:39.268415 27534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:29:39.269309 27534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790321110323796-5753-0/minicluster-data/master-0/data/instance:
uuid: "97231315542141a681a9efa021d6cbee"
format_stamp: "Formatted at 2026-09-25 07:29:39 on dist-test-slave-1drq"
server_key: "4375eaea18c5bb13b59f58940b4edd6d"
server_key_iv: "37769a57489b4de6d097cfa39f96cc6a"
server_key_version: "encryptionkey@0"
I20260925 07:29:39.269582 27534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790321110323796-5753-0/minicluster-data/master-0/wal/instance:
uuid: "97231315542141a681a9efa021d6cbee"
format_stamp: "Formatted at 2026-09-25 07:29:39 on dist-test-slave-1drq"
server_key: "4375eaea18c5bb13b59f58940b4edd6d"
server_key_iv: "37769a57489b4de6d097cfa39f96cc6a"
server_key_version: "encryptionkey@0"
I20260925 07:29:39.270676 27534 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 07:29:39.271304 27549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:39.271494 27534 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:29:39.271595 27534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "97231315542141a681a9efa021d6cbee"
format_stamp: "Formatted at 2026-09-25 07:29:39 on dist-test-slave-1drq"
server_key: "4375eaea18c5bb13b59f58940b4edd6d"
server_key_iv: "37769a57489b4de6d097cfa39f96cc6a"
server_key_version: "encryptionkey@0"
I20260925 07:29:39.271659 27534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:29:39.274529 27534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:29:39.276677 27534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:29:39.276808 27534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:29:39.279969 27534 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.126:36213
I20260925 07:29:39.280047 27601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.126:36213 every 8 connection(s)
I20260925 07:29:39.280274 27534 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:29:39.281307 27602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:29:39.283318 27602 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97231315542141a681a9efa021d6cbee: Bootstrap starting.
I20260925 07:29:39.283887 27602 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97231315542141a681a9efa021d6cbee: Neither blocks nor log segments found. Creating new log.
I20260925 07:29:39.284134 27602 log.cc:824] T 00000000000000000000000000000000 P 97231315542141a681a9efa021d6cbee: Log is configured to *not* fsync() on all Append() calls
I20260925 07:29:39.284732 27602 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97231315542141a681a9efa021d6cbee: No bootstrap required, opened a new log
I20260925 07:29:39.285442 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 27534
I20260925 07:29:39.285567 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790321110323796-5753-0/minicluster-data/master-0/wal/instance
I20260925 07:29:39.285748 5753 external_mini_cluster.cc:1428] Setting key 695fc0c032ef91399fb572be2164f747
I20260925 07:29:39.286509 27602 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97231315542141a681a9efa021d6cbee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97231315542141a681a9efa021d6cbee" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 36213 } }
I20260925 07:29:39.286667 27602 raft_consensus.cc:374] T 00000000000000000000000000000000 P 97231315542141a681a9efa021d6cbee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:29:39.286705 27602 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97231315542141a681a9efa021d6cbee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97231315542141a681a9efa021d6cbee, State: Initialized, Role: FOLLOWER
I20260925 07:29:39.286814 27602 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97231315542141a681a9efa021d6cbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97231315542141a681a9efa021d6cbee" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 36213 } }
I20260925 07:29:39.286893 27602 raft_consensus.cc:388] T 00000000000000000000000000000000 P 97231315542141a681a9efa021d6cbee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 07:29:39.286914 27602 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97231315542141a681a9efa021d6cbee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 07:29:39.286969 27602 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97231315542141a681a9efa021d6cbee [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:29:39.287678 27606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:39.286173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52363 (local address 127.5.158.126:36213)
0925 07:29:39.286329 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:39.286335 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:39.286361 (+ 26us) server_negotiation.cc:407] Connection header received
0925 07:29:39.286474 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:39.286478 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:39.286500 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:39.286591 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:39.287236 (+ 645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:39.287239 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:39.287239 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:29:39.287242 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:39.287274 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:29:39.287283 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:39.287330 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:39.287430 (+ 100us) server_negotiation.cc:299] Negotiation successful
0925 07:29:39.287466 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":87,"threads_started":1}
I20260925 07:29:39.288040 27602 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97231315542141a681a9efa021d6cbee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97231315542141a681a9efa021d6cbee" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 36213 } }
I20260925 07:29:39.288160 27602 leader_election.cc:302] T 00000000000000000000000000000000 P 97231315542141a681a9efa021d6cbee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 97231315542141a681a9efa021d6cbee; no voters:
I20260925 07:29:39.288340 27602 leader_election.cc:288] T 00000000000000000000000000000000 P 97231315542141a681a9efa021d6cbee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 07:29:39.288357 27607 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97231315542141a681a9efa021d6cbee [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:29:39.288442 5753 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 07:29:39.288566 27607 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97231315542141a681a9efa021d6cbee [term 1 LEADER]: Becoming Leader. State: Replica: 97231315542141a681a9efa021d6cbee, State: Running, Role: LEADER
I20260925 07:29:39.288590 27602 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97231315542141a681a9efa021d6cbee [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:29:39.289829 27607 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97231315542141a681a9efa021d6cbee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97231315542141a681a9efa021d6cbee" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 36213 } }
I20260925 07:29:39.290381 27609 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97231315542141a681a9efa021d6cbee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "97231315542141a681a9efa021d6cbee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97231315542141a681a9efa021d6cbee" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 36213 } } }
W20260925 07:29:39.290762 27626 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 97231315542141a681a9efa021d6cbee: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260925 07:29:39.290377 27608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97231315542141a681a9efa021d6cbee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97231315542141a681a9efa021d6cbee. Latest consensus state: current_term: 1 leader_uuid: "97231315542141a681a9efa021d6cbee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97231315542141a681a9efa021d6cbee" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 36213 } } }
W20260925 07:29:39.290810 27626 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 07:29:39.290827 27608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97231315542141a681a9efa021d6cbee [sys.catalog]: This master's current role is: LEADER
I20260925 07:29:39.290814 27609 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97231315542141a681a9efa021d6cbee [sys.catalog]: This master's current role is: LEADER
I20260925 07:29:39.291093 27606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:39.289748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60177 (local address 127.5.158.126:36213)
0925 07:29:39.289773 (+ 25us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:39.289774 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:39.289949 (+ 175us) server_negotiation.cc:407] Connection header received
0925 07:29:39.289971 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:39.289972 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:39.289975 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:39.290039 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:39.290851 (+ 812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:39.290852 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:39.290852 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:29:39.290856 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:39.290870 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:29:39.290874 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:39.290923 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:39.290983 (+ 60us) server_negotiation.cc:299] Negotiation successful
0925 07:29:39.290998 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12}
I20260925 07:29:39.291028 27629 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 07:29:39.291325 27629 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 07:29:39.292753 27629 catalog_manager.cc:1347] Generated new cluster ID: 6513c2481a924e0d9e3702ed865fdb5d
I20260925 07:29:39.292805 27629 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 07:29:39.316412 27629 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 07:29:39.316563 27629 catalog_manager.cc:1504] Loading token signing keys...
I20260925 07:29:39.323877 27629 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 97231315542141a681a9efa021d6cbee: Generated new TSK 0
I20260925 07:29:39.324024 27629 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 07:29:39.327230 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 27534
2026-09-25T07:29:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1191 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790321110323796-5753-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 07:29:39 dist-test-slave-1drq krb5kdc[27633](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 07:29:39 dist-test-slave-1drq krb5kdc[27633](info): set up 2 sockets
Sep 25 07:29:39 dist-test-slave-1drq krb5kdc[27633](info): commencing operation
krb5kdc: starting...
W20260925 07:29:40.284863 5753 mini_kdc.cc:121] Time spent starting KDC: real 0.940s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Sep 25 07:29:40 dist-test-slave-1drq krb5kdc[27633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T07:29:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T07:29:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
I20260925 07:29:40.391968 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:44841
--webserver_interface=127.5.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:42483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.158.126:44841
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:29:40.464303 27646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:29:40.464504 27646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:29:40.464540 27646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:29:40.465575 27646 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:29:40.465641 27646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:29:40.465664 27646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:29:40.465696 27646 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:29:40.465720 27646 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:29:40.466990 27646 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:42483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790321110323796-5753-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.158.126:44841
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:44841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27646
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:29:40.467229 27646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:29:40.467439 27646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:29:40.471769 27651 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:40.471792 27652 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:40.471906 27646 server_base.cc:1034] running on GCE node
W20260925 07:29:40.471792 27654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:40.472196 27646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:29:40.472500 27646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:29:40.473639 27646 hybrid_clock.cc:648] HybridClock initialized: now 1790321380473630 us; error 27 us; skew 500 ppm
Sep 25 07:29:40 dist-test-slave-1drq krb5kdc[27633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321380, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:29:40.481938 27646 init.cc:377] Logged in from keytab as kudu/127.5.158.126@KRBTEST.COM (short username kudu)
I20260925 07:29:40.482336 27646 webserver.cc:460] Webserver started at http://127.5.158.126:40707/ using document root <none> and password file <none>
I20260925 07:29:40.482569 27646 fs_manager.cc:359] Metadata directory not provided
I20260925 07:29:40.482635 27646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:29:40.482774 27646 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:29:40.483641 27646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790321110323796-5753-0/minicluster-data/master-0/data/instance:
uuid: "f8e8431c3bbd4998ade11536397bf37e"
format_stamp: "Formatted at 2026-09-25 07:29:40 on dist-test-slave-1drq"
server_key: "a9fec7b7f4e43b17ff7dda3546c5c294"
server_key_iv: "6de9af24a48de5aaa5d4c911f426b19c"
server_key_version: "encryptionkey@0"
I20260925 07:29:40.483983 27646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790321110323796-5753-0/minicluster-data/master-0/wal/instance:
uuid: "f8e8431c3bbd4998ade11536397bf37e"
format_stamp: "Formatted at 2026-09-25 07:29:40 on dist-test-slave-1drq"
server_key: "a9fec7b7f4e43b17ff7dda3546c5c294"
server_key_iv: "6de9af24a48de5aaa5d4c911f426b19c"
server_key_version: "encryptionkey@0"
I20260925 07:29:40.485245 27646 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 07:29:40.485980 27661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:40.486135 27646 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:29:40.486230 27646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "f8e8431c3bbd4998ade11536397bf37e"
format_stamp: "Formatted at 2026-09-25 07:29:40 on dist-test-slave-1drq"
server_key: "a9fec7b7f4e43b17ff7dda3546c5c294"
server_key_iv: "6de9af24a48de5aaa5d4c911f426b19c"
server_key_version: "encryptionkey@0"
I20260925 07:29:40.486310 27646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:29:40.501782 27646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:29:40.503454 27646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:29:40.503631 27646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:29:40.507702 27646 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.126:44841
I20260925 07:29:40.507736 27713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.126:44841 every 8 connection(s)
I20260925 07:29:40.508050 27646 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:29:40.508858 27714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:29:40.510813 27714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8e8431c3bbd4998ade11536397bf37e: Bootstrap starting.
I20260925 07:29:40.511363 27714 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f8e8431c3bbd4998ade11536397bf37e: Neither blocks nor log segments found. Creating new log.
I20260925 07:29:40.511650 27714 log.cc:824] T 00000000000000000000000000000000 P f8e8431c3bbd4998ade11536397bf37e: Log is configured to *not* fsync() on all Append() calls
I20260925 07:29:40.512310 27714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8e8431c3bbd4998ade11536397bf37e: No bootstrap required, opened a new log
I20260925 07:29:40.513826 27714 raft_consensus.cc:348] T 00000000000000000000000000000000 P f8e8431c3bbd4998ade11536397bf37e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8e8431c3bbd4998ade11536397bf37e" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 44841 } }
I20260925 07:29:40.513944 27714 raft_consensus.cc:374] T 00000000000000000000000000000000 P f8e8431c3bbd4998ade11536397bf37e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:29:40.513964 27714 raft_consensus.cc:729] T 00000000000000000000000000000000 P f8e8431c3bbd4998ade11536397bf37e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8e8431c3bbd4998ade11536397bf37e, State: Initialized, Role: FOLLOWER
I20260925 07:29:40.514122 27714 consensus_queue.cc:260] T 00000000000000000000000000000000 P f8e8431c3bbd4998ade11536397bf37e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8e8431c3bbd4998ade11536397bf37e" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 44841 } }
I20260925 07:29:40.514205 27714 raft_consensus.cc:388] T 00000000000000000000000000000000 P f8e8431c3bbd4998ade11536397bf37e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 07:29:40.514227 27714 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8e8431c3bbd4998ade11536397bf37e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 07:29:40.514273 27714 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f8e8431c3bbd4998ade11536397bf37e [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:29:40.514995 27714 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8e8431c3bbd4998ade11536397bf37e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8e8431c3bbd4998ade11536397bf37e" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 44841 } }
I20260925 07:29:40.515072 27714 leader_election.cc:302] T 00000000000000000000000000000000 P f8e8431c3bbd4998ade11536397bf37e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f8e8431c3bbd4998ade11536397bf37e; no voters:
I20260925 07:29:40.515231 27714 leader_election.cc:288] T 00000000000000000000000000000000 P f8e8431c3bbd4998ade11536397bf37e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 07:29:40.515271 27717 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f8e8431c3bbd4998ade11536397bf37e [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:29:40.515530 27717 raft_consensus.cc:686] T 00000000000000000000000000000000 P f8e8431c3bbd4998ade11536397bf37e [term 1 LEADER]: Becoming Leader. State: Replica: f8e8431c3bbd4998ade11536397bf37e, State: Running, Role: LEADER
I20260925 07:29:40.515632 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 27646
I20260925 07:29:40.515625 27717 consensus_queue.cc:237] T 00000000000000000000000000000000 P f8e8431c3bbd4998ade11536397bf37e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8e8431c3bbd4998ade11536397bf37e" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 44841 } }
I20260925 07:29:40.515736 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790321110323796-5753-0/minicluster-data/master-0/wal/instance
I20260925 07:29:40.515913 5753 external_mini_cluster.cc:1428] Setting key 83d4ed9ddece113dd557f01f6cefe8be
I20260925 07:29:40.516021 27719 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8e8431c3bbd4998ade11536397bf37e [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8e8431c3bbd4998ade11536397bf37e. Latest consensus state: current_term: 1 leader_uuid: "f8e8431c3bbd4998ade11536397bf37e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8e8431c3bbd4998ade11536397bf37e" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 44841 } } }
I20260925 07:29:40.516220 27719 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8e8431c3bbd4998ade11536397bf37e [sys.catalog]: This master's current role is: LEADER
I20260925 07:29:40.516029 27718 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8e8431c3bbd4998ade11536397bf37e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f8e8431c3bbd4998ade11536397bf37e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8e8431c3bbd4998ade11536397bf37e" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 44841 } } }
I20260925 07:29:40.516106 27714 sys_catalog.cc:564] T 00000000000000000000000000000000 P f8e8431c3bbd4998ade11536397bf37e [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:29:40.516490 27718 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8e8431c3bbd4998ade11536397bf37e [sys.catalog]: This master's current role is: LEADER
I20260925 07:29:40.516705 27722 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 07:29:40.517235 27722 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 07:29:40.518801 27722 catalog_manager.cc:1347] Generated new cluster ID: 2c015c48c0ec4bca9f99e033f8f0435d
I20260925 07:29:40.518853 27722 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 07:29:40 dist-test-slave-1drq krb5kdc[27633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:29:40.526384 27723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:40.516362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37353 (local address 127.5.158.126:44841)
0925 07:29:40.517172 (+ 810us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:40.517182 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:40.517221 (+ 39us) server_negotiation.cc:407] Connection header received
0925 07:29:40.517327 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:40.517334 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:40.517388 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:40.518088 (+ 700us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:40.518428 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:40.519426 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:40.520254 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:40.520682 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:40.520817 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:40.523128 (+ 2311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:40.523132 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:40.523132 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:29:40.523143 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:40.524776 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:40.525223 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:40.525227 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:40.525227 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:40.525327 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:40.525551 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:40.525552 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:40.525553 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:40.525789 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:40.525983 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:40.526074 (+ 91us) server_negotiation.cc:299] Negotiation successful
0925 07:29:40.526208 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":760,"thread_start_us":82,"threads_started":1}
I20260925 07:29:40.527281 5753 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 07:29:40.527930 27722 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 07:29:40.528553 27722 catalog_manager.cc:1504] Loading token signing keys...
I20260925 07:29:40.534945 27723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:40.528868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34055 (local address 127.5.158.126:44841)
0925 07:29:40.528899 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:40.528901 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:40.529237 (+ 336us) server_negotiation.cc:407] Connection header received
0925 07:29:40.529305 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:40.529306 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:40.529318 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:40.529369 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:40.529890 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:40.530354 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:40.531065 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:40.531293 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:40.531373 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:40.532529 (+ 1156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:40.532530 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:40.532530 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:29:40.532537 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:40.533861 (+ 1324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:40.534203 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:40.534204 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:40.534204 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:40.534251 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:40.534450 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:40.534451 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:40.534451 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:40.534549 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:40.534655 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:40.534741 (+ 86us) server_negotiation.cc:299] Negotiation successful
0925 07:29:40.534806 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260925 07:29:40.535885 27722 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f8e8431c3bbd4998ade11536397bf37e: Generated new TSK 0
I20260925 07:29:40.536078 27722 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 07:29:40.542239 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 27646
2026-09-25T07:29:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1215 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790321110323796-5753-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 07:29:40 dist-test-slave-1drq krb5kdc[27745](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 07:29:40 dist-test-slave-1drq krb5kdc[27745](info): set up 2 sockets
Sep 25 07:29:40 dist-test-slave-1drq krb5kdc[27745](info): commencing operation
krb5kdc: starting...
W20260925 07:29:41.497326 5753 mini_kdc.cc:121] Time spent starting KDC: real 0.938s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Sep 25 07:29:41 dist-test-slave-1drq krb5kdc[27745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321381, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T07:29:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T07:29:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
I20260925 07:29:41.617800 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:41123
--webserver_interface=127.5.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:37577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.158.126:41123
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:29:41.688743 27758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:29:41.688943 27758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:29:41.688978 27758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:29:41.690058 27758 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:29:41.690124 27758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:29:41.690146 27758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:29:41.690173 27758 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:29:41.690197 27758 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:29:41.691555 27758 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:37577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790321110323796-5753-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.158.126:41123
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:41123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27758
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:29:41.691812 27758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:29:41.692029 27758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:29:41.696581 27764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:41.696728 27758 server_base.cc:1034] running on GCE node
W20260925 07:29:41.696642 27766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:41.696594 27763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:41.697005 27758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:29:41.697307 27758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:29:41.698462 27758 hybrid_clock.cc:648] HybridClock initialized: now 1790321381698447 us; error 35 us; skew 500 ppm
Sep 25 07:29:41 dist-test-slave-1drq krb5kdc[27745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321381, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:29:41.700723 27758 init.cc:377] Logged in from keytab as kudu/127.5.158.126@KRBTEST.COM (short username kudu)
I20260925 07:29:41.701162 27758 webserver.cc:460] Webserver started at http://127.5.158.126:33089/ using document root <none> and password file <none>
I20260925 07:29:41.701390 27758 fs_manager.cc:359] Metadata directory not provided
I20260925 07:29:41.701432 27758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:29:41.701582 27758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:29:41.702529 27758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790321110323796-5753-0/minicluster-data/master-0/data/instance:
uuid: "2784a3f915d14db6a22bb2ee43f5ccd2"
format_stamp: "Formatted at 2026-09-25 07:29:41 on dist-test-slave-1drq"
server_key: "eebede796d205244344e56ef93b56d30"
server_key_iv: "7e9f9ec307af06be186972925d731883"
server_key_version: "encryptionkey@0"
I20260925 07:29:41.702845 27758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790321110323796-5753-0/minicluster-data/master-0/wal/instance:
uuid: "2784a3f915d14db6a22bb2ee43f5ccd2"
format_stamp: "Formatted at 2026-09-25 07:29:41 on dist-test-slave-1drq"
server_key: "eebede796d205244344e56ef93b56d30"
server_key_iv: "7e9f9ec307af06be186972925d731883"
server_key_version: "encryptionkey@0"
I20260925 07:29:41.704038 27758 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 07:29:41.704665 27773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:41.704820 27758 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:29:41.704916 27758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "2784a3f915d14db6a22bb2ee43f5ccd2"
format_stamp: "Formatted at 2026-09-25 07:29:41 on dist-test-slave-1drq"
server_key: "eebede796d205244344e56ef93b56d30"
server_key_iv: "7e9f9ec307af06be186972925d731883"
server_key_version: "encryptionkey@0"
I20260925 07:29:41.704993 27758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:29:41.713843 27758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:29:41.715396 27758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:29:41.715554 27758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:29:41.718986 27758 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.126:41123
I20260925 07:29:41.719015 27825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.126:41123 every 8 connection(s)
I20260925 07:29:41.719324 27758 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:29:41.720131 27826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:29:41.720960 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 27758
I20260925 07:29:41.721140 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790321110323796-5753-0/minicluster-data/master-0/wal/instance
I20260925 07:29:41.721338 5753 external_mini_cluster.cc:1428] Setting key c494f453470a786e1e647cc5b99f471a
I20260925 07:29:41.722543 27826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2784a3f915d14db6a22bb2ee43f5ccd2: Bootstrap starting.
I20260925 07:29:41.723116 27826 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2784a3f915d14db6a22bb2ee43f5ccd2: Neither blocks nor log segments found. Creating new log.
I20260925 07:29:41.723395 27826 log.cc:824] T 00000000000000000000000000000000 P 2784a3f915d14db6a22bb2ee43f5ccd2: Log is configured to *not* fsync() on all Append() calls
I20260925 07:29:41.724031 27826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2784a3f915d14db6a22bb2ee43f5ccd2: No bootstrap required, opened a new log
I20260925 07:29:41.725699 27826 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2784a3f915d14db6a22bb2ee43f5ccd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2784a3f915d14db6a22bb2ee43f5ccd2" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 41123 } }
I20260925 07:29:41.725836 27826 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2784a3f915d14db6a22bb2ee43f5ccd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:29:41.725883 27826 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2784a3f915d14db6a22bb2ee43f5ccd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2784a3f915d14db6a22bb2ee43f5ccd2, State: Initialized, Role: FOLLOWER
I20260925 07:29:41.726002 27826 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2784a3f915d14db6a22bb2ee43f5ccd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2784a3f915d14db6a22bb2ee43f5ccd2" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 41123 } }
I20260925 07:29:41.726074 27826 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2784a3f915d14db6a22bb2ee43f5ccd2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 07:29:41.726111 27826 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2784a3f915d14db6a22bb2ee43f5ccd2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 07:29:41.726152 27826 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2784a3f915d14db6a22bb2ee43f5ccd2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:29:41.726357 27829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:41.721814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35827 (local address 127.5.158.126:41123)
0925 07:29:41.721986 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:41.721994 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:41.722021 (+ 27us) server_negotiation.cc:407] Connection header received
0925 07:29:41.722394 (+ 373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:41.722399 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:41.722439 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:41.722544 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:41.722778 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:41.723691 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:41.724527 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:41.724918 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:41.725037 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:41.725682 (+ 645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:41.725686 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:41.725686 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:29:41.725828 (+ 142us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:41.725853 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:29:41.725886 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:41.725950 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:41.726018 (+ 68us) server_negotiation.cc:299] Negotiation successful
0925 07:29:41.726146 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":89,"threads_started":1}
I20260925 07:29:41.726742 27826 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2784a3f915d14db6a22bb2ee43f5ccd2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2784a3f915d14db6a22bb2ee43f5ccd2" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 41123 } }
I20260925 07:29:41.726850 27826 leader_election.cc:302] T 00000000000000000000000000000000 P 2784a3f915d14db6a22bb2ee43f5ccd2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2784a3f915d14db6a22bb2ee43f5ccd2; no voters:
I20260925 07:29:41.727020 27826 leader_election.cc:288] T 00000000000000000000000000000000 P 2784a3f915d14db6a22bb2ee43f5ccd2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 07:29:41.727043 27831 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2784a3f915d14db6a22bb2ee43f5ccd2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:29:41.727190 27831 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2784a3f915d14db6a22bb2ee43f5ccd2 [term 1 LEADER]: Becoming Leader. State: Replica: 2784a3f915d14db6a22bb2ee43f5ccd2, State: Running, Role: LEADER
I20260925 07:29:41.727182 5753 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 07:29:41.727250 27826 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2784a3f915d14db6a22bb2ee43f5ccd2 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:29:41.727308 27831 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2784a3f915d14db6a22bb2ee43f5ccd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2784a3f915d14db6a22bb2ee43f5ccd2" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 41123 } }
I20260925 07:29:41.727571 27831 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2784a3f915d14db6a22bb2ee43f5ccd2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2784a3f915d14db6a22bb2ee43f5ccd2. Latest consensus state: current_term: 1 leader_uuid: "2784a3f915d14db6a22bb2ee43f5ccd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2784a3f915d14db6a22bb2ee43f5ccd2" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 41123 } } }
I20260925 07:29:41.727643 27831 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2784a3f915d14db6a22bb2ee43f5ccd2 [sys.catalog]: This master's current role is: LEADER
I20260925 07:29:41.727576 27833 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2784a3f915d14db6a22bb2ee43f5ccd2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2784a3f915d14db6a22bb2ee43f5ccd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2784a3f915d14db6a22bb2ee43f5ccd2" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 41123 } } }
I20260925 07:29:41.727767 27833 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2784a3f915d14db6a22bb2ee43f5ccd2 [sys.catalog]: This master's current role is: LEADER
W20260925 07:29:41.729207 27851 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2784a3f915d14db6a22bb2ee43f5ccd2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 07:29:41.729281 27851 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 07:29:41.729344 27835 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 07:29:41.729508 27835 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 07:29:41.731025 27835 catalog_manager.cc:1347] Generated new cluster ID: 4ffbadff55bb4553b7275966a6fb7c50
I20260925 07:29:41.731068 27835 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 07:29:41.731642 27829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:41.728648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44049 (local address 127.5.158.126:41123)
0925 07:29:41.728725 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:41.728727 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:41.728734 (+ 7us) server_negotiation.cc:407] Connection header received
0925 07:29:41.728820 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:41.728821 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:41.728828 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:41.728851 (+ 23us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:41.729017 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:41.729622 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:41.730531 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:41.730779 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:41.730846 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:41.731413 (+ 567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:41.731414 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:41.731414 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:29:41.731418 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:41.731429 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:29:41.731433 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:41.731474 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:41.731537 (+ 63us) server_negotiation.cc:299] Negotiation successful
0925 07:29:41.731555 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260925 07:29:41.742000 27835 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 07:29:41.742465 27835 catalog_manager.cc:1504] Loading token signing keys...
I20260925 07:29:41.748571 27835 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2784a3f915d14db6a22bb2ee43f5ccd2: Generated new TSK 0
I20260925 07:29:41.748705 27835 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 07:29:41.751350 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 27758
2026-09-25T07:29:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1208 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790321110323796-5753-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 07:29:41 dist-test-slave-1drq krb5kdc[27857](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 07:29:41 dist-test-slave-1drq krb5kdc[27857](info): set up 2 sockets
Sep 25 07:29:41 dist-test-slave-1drq krb5kdc[27857](info): commencing operation
krb5kdc: starting...
W20260925 07:29:42.693010 5753 mini_kdc.cc:121] Time spent starting KDC: real 0.925s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Sep 25 07:29:42 dist-test-slave-1drq krb5kdc[27857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321382, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T07:29:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T07:29:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
I20260925 07:29:42.799383 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:43973
--webserver_interface=127.5.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:42117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.158.126:43973
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:29:42.871184 27870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:29:42.871348 27870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:29:42.871368 27870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:29:42.872575 27870 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:29:42.872625 27870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:29:42.872638 27870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:29:42.872651 27870 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:29:42.872663 27870 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:29:42.873927 27870 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:42117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790321110323796-5753-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.158.126:43973
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:43973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27870
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:29:42.874070 27870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:29:42.874262 27870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 07:29:42.878947 27870 server_base.cc:1034] running on GCE node
W20260925 07:29:42.878932 27878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:42.878917 27876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:42.878923 27875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:42.879495 27870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:29:42.879721 27870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:29:42.880877 27870 hybrid_clock.cc:648] HybridClock initialized: now 1790321382880854 us; error 36 us; skew 500 ppm
Sep 25 07:29:42 dist-test-slave-1drq krb5kdc[27857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321382, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:29:42.883194 27870 init.cc:377] Logged in from keytab as kudu/127.5.158.126@KRBTEST.COM (short username kudu)
I20260925 07:29:42.883597 27870 webserver.cc:460] Webserver started at http://127.5.158.126:41101/ using document root <none> and password file <none>
I20260925 07:29:42.883762 27870 fs_manager.cc:359] Metadata directory not provided
I20260925 07:29:42.883800 27870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:29:42.883877 27870 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:29:42.884953 27870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790321110323796-5753-0/minicluster-data/master-0/data/instance:
uuid: "96a21b75e19c48b380922062a26b0db8"
format_stamp: "Formatted at 2026-09-25 07:29:42 on dist-test-slave-1drq"
server_key: "c208a0ce0eb26365e5c6a8c03c5f5366"
server_key_iv: "b3cd6f8c32a1875f5b6bcf2f771326f8"
server_key_version: "encryptionkey@0"
I20260925 07:29:42.885301 27870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790321110323796-5753-0/minicluster-data/master-0/wal/instance:
uuid: "96a21b75e19c48b380922062a26b0db8"
format_stamp: "Formatted at 2026-09-25 07:29:42 on dist-test-slave-1drq"
server_key: "c208a0ce0eb26365e5c6a8c03c5f5366"
server_key_iv: "b3cd6f8c32a1875f5b6bcf2f771326f8"
server_key_version: "encryptionkey@0"
I20260925 07:29:42.886508 27870 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 07:29:42.887164 27885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:42.887331 27870 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 07:29:42.887426 27870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "96a21b75e19c48b380922062a26b0db8"
format_stamp: "Formatted at 2026-09-25 07:29:42 on dist-test-slave-1drq"
server_key: "c208a0ce0eb26365e5c6a8c03c5f5366"
server_key_iv: "b3cd6f8c32a1875f5b6bcf2f771326f8"
server_key_version: "encryptionkey@0"
I20260925 07:29:42.887506 27870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:29:42.890537 27870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:29:42.892199 27870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:29:42.892349 27870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:29:42.895967 27870 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.126:43973
I20260925 07:29:42.895991 27937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.126:43973 every 8 connection(s)
I20260925 07:29:42.896257 27870 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:29:42.897013 27938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:29:42.898880 27938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96a21b75e19c48b380922062a26b0db8: Bootstrap starting.
I20260925 07:29:42.899354 27938 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 96a21b75e19c48b380922062a26b0db8: Neither blocks nor log segments found. Creating new log.
I20260925 07:29:42.899611 27938 log.cc:824] T 00000000000000000000000000000000 P 96a21b75e19c48b380922062a26b0db8: Log is configured to *not* fsync() on all Append() calls
I20260925 07:29:42.900187 27938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96a21b75e19c48b380922062a26b0db8: No bootstrap required, opened a new log
I20260925 07:29:42.901783 27938 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96a21b75e19c48b380922062a26b0db8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96a21b75e19c48b380922062a26b0db8" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 43973 } }
I20260925 07:29:42.901903 27938 raft_consensus.cc:374] T 00000000000000000000000000000000 P 96a21b75e19c48b380922062a26b0db8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 07:29:42.901926 27938 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96a21b75e19c48b380922062a26b0db8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a21b75e19c48b380922062a26b0db8, State: Initialized, Role: FOLLOWER
I20260925 07:29:42.902115 27938 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96a21b75e19c48b380922062a26b0db8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96a21b75e19c48b380922062a26b0db8" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 43973 } }
I20260925 07:29:42.902176 27938 raft_consensus.cc:388] T 00000000000000000000000000000000 P 96a21b75e19c48b380922062a26b0db8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 07:29:42.902197 27938 raft_consensus.cc:482] T 00000000000000000000000000000000 P 96a21b75e19c48b380922062a26b0db8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 07:29:42.902259 27938 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96a21b75e19c48b380922062a26b0db8 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:29:42.902915 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 27870
I20260925 07:29:42.903043 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790321110323796-5753-0/minicluster-data/master-0/wal/instance
I20260925 07:29:42.903124 27938 raft_consensus.cc:504] T 00000000000000000000000000000000 P 96a21b75e19c48b380922062a26b0db8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96a21b75e19c48b380922062a26b0db8" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 43973 } }
I20260925 07:29:42.903239 27938 leader_election.cc:302] T 00000000000000000000000000000000 P 96a21b75e19c48b380922062a26b0db8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 96a21b75e19c48b380922062a26b0db8; no voters:
I20260925 07:29:42.903263 5753 external_mini_cluster.cc:1428] Setting key e8228ae42498494fcfec82ea1675794c
I20260925 07:29:42.903404 27938 leader_election.cc:288] T 00000000000000000000000000000000 P 96a21b75e19c48b380922062a26b0db8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 07:29:42.903450 27941 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 96a21b75e19c48b380922062a26b0db8 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:29:42.903594 27941 raft_consensus.cc:686] T 00000000000000000000000000000000 P 96a21b75e19c48b380922062a26b0db8 [term 1 LEADER]: Becoming Leader. State: Replica: 96a21b75e19c48b380922062a26b0db8, State: Running, Role: LEADER
I20260925 07:29:42.903644 27938 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96a21b75e19c48b380922062a26b0db8 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:29:42.903729 27941 consensus_queue.cc:237] T 00000000000000000000000000000000 P 96a21b75e19c48b380922062a26b0db8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96a21b75e19c48b380922062a26b0db8" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 43973 } }
I20260925 07:29:42.904156 27942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96a21b75e19c48b380922062a26b0db8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "96a21b75e19c48b380922062a26b0db8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96a21b75e19c48b380922062a26b0db8" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 43973 } } }
I20260925 07:29:42.904245 27942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96a21b75e19c48b380922062a26b0db8 [sys.catalog]: This master's current role is: LEADER
I20260925 07:29:42.904152 27943 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96a21b75e19c48b380922062a26b0db8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96a21b75e19c48b380922062a26b0db8. Latest consensus state: current_term: 1 leader_uuid: "96a21b75e19c48b380922062a26b0db8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96a21b75e19c48b380922062a26b0db8" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 43973 } } }
I20260925 07:29:42.904421 27943 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96a21b75e19c48b380922062a26b0db8 [sys.catalog]: This master's current role is: LEADER
I20260925 07:29:42.904847 27948 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 07:29:42.905369 27948 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 07:29:42.905645 27945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:42.903876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55317 (local address 127.5.158.126:43973)
0925 07:29:42.904024 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:42.904035 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:42.904065 (+ 30us) server_negotiation.cc:407] Connection header received
0925 07:29:42.904310 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:42.904318 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:42.904347 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:42.904520 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:42.905169 (+ 649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:42.905172 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:42.905172 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:29:42.905177 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:42.905219 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:29:42.905232 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:42.905437 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:42.905465 (+ 28us) server_negotiation.cc:299] Negotiation successful
0925 07:29:42.905499 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":76,"threads_started":1}
I20260925 07:29:42.906664 5753 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 07:29:42.907776 27948 catalog_manager.cc:1347] Generated new cluster ID: 79c3e9e8ac754a0d9c9b276d1ef1367c
I20260925 07:29:42.907827 27948 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 07:29:42.914011 27945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:42.907897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45703 (local address 127.5.158.126:43973)
0925 07:29:42.907928 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:42.907930 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:42.913244 (+ 5314us) server_negotiation.cc:407] Connection header received
0925 07:29:42.913271 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:42.913272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:42.913276 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:42.913330 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:42.913822 (+ 492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:42.913823 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:42.913824 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 07:29:42.913827 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:42.913839 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 07:29:42.913842 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:42.913907 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:42.913949 (+ 42us) server_negotiation.cc:299] Negotiation successful
0925 07:29:42.913960 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 07:29:42.929307 27948 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 07:29:42.929435 27948 catalog_manager.cc:1504] Loading token signing keys...
I20260925 07:29:42.935014 27948 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 96a21b75e19c48b380922062a26b0db8: Generated new TSK 0
I20260925 07:29:42.935182 27948 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 07:29:42.940030 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 27870
2026-09-25T07:29:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1189 ms)
[----------] 18 tests from AuthTokenIssuingTest (21540 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 07:29:42 dist-test-slave-1drq krb5kdc[27969](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 07:29:42 dist-test-slave-1drq krb5kdc[27969](info): set up 2 sockets
Sep 25 07:29:42 dist-test-slave-1drq krb5kdc[27969](info): commencing operation
krb5kdc: starting...
W20260925 07:29:43.893106 5753 mini_kdc.cc:121] Time spent starting KDC: real 0.935s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Sep 25 07:29:43 dist-test-slave-1drq krb5kdc[27969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321383, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T07:29:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T07:29:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
I20260925 07:29:43.999439 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:45699
--webserver_interface=127.5.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:44165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.158.126:45699,127.5.158.125:33185,127.5.158.124:37331
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:29:44.074751 27982 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 07:29:44.074936 27982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:29:44.074971 27982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:29:44.074990 27982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:29:44.075930 27982 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:29:44.075964 27982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:29:44.075978 27982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:29:44.075990 27982 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:29:44.076002 27982 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:29:44.077350 27982 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:44165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.158.126:45699,127.5.158.125:33185,127.5.158.124:37331
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:45699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27982
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:29:44.077531 27982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:29:44.077685 27982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:29:44.082079 27990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:44.082146 27982 server_base.cc:1034] running on GCE node
W20260925 07:29:44.082106 27987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:44.082101 27988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:44.082581 27982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:29:44.082891 27982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:29:44.084054 27982 hybrid_clock.cc:648] HybridClock initialized: now 1790321384084039 us; error 45 us; skew 500 ppm
Sep 25 07:29:44 dist-test-slave-1drq krb5kdc[27969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321384, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:29:44.086352 27982 init.cc:377] Logged in from keytab as kudu/127.5.158.126@KRBTEST.COM (short username kudu)
I20260925 07:29:44.086684 27982 webserver.cc:460] Webserver started at http://127.5.158.126:40031/ using document root <none> and password file <none>
I20260925 07:29:44.086900 27982 fs_manager.cc:359] Metadata directory not provided
I20260925 07:29:44.086953 27982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:29:44.087155 27982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:29:44.088106 27982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/minicluster-data/master-0/data/instance:
uuid: "08be0716dc1741b3b8ea2b59242728b8"
format_stamp: "Formatted at 2026-09-25 07:29:44 on dist-test-slave-1drq"
server_key: "327ea07ac94999de1987d48c3c7a4b8f"
server_key_iv: "77d78fd4f036a6cf1aa1ad99d0ef4dac"
server_key_version: "encryptionkey@0"
I20260925 07:29:44.088434 27982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/minicluster-data/master-0/wal/instance:
uuid: "08be0716dc1741b3b8ea2b59242728b8"
format_stamp: "Formatted at 2026-09-25 07:29:44 on dist-test-slave-1drq"
server_key: "327ea07ac94999de1987d48c3c7a4b8f"
server_key_iv: "77d78fd4f036a6cf1aa1ad99d0ef4dac"
server_key_version: "encryptionkey@0"
I20260925 07:29:44.089743 27982 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260925 07:29:44.090410 27997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:44.090574 27982 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:29:44.090670 27982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "08be0716dc1741b3b8ea2b59242728b8"
format_stamp: "Formatted at 2026-09-25 07:29:44 on dist-test-slave-1drq"
server_key: "327ea07ac94999de1987d48c3c7a4b8f"
server_key_iv: "77d78fd4f036a6cf1aa1ad99d0ef4dac"
server_key_version: "encryptionkey@0"
I20260925 07:29:44.090760 27982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:29:44.101485 27982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:29:44.103091 27982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:29:44.103235 27982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:29:44.107060 27982 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.126:45699
I20260925 07:29:44.107092 28049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.126:45699 every 8 connection(s)
I20260925 07:29:44.107378 27982 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:29:44.108263 28050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:29:44.109724 28050 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.158.126" port: 45699 } has no permanent_uuid. Determining permanent_uuid...
I20260925 07:29:44.113495 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 27982
I20260925 07:29:44.113602 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/minicluster-data/master-0/wal/instance
I20260925 07:29:44.113786 5753 external_mini_cluster.cc:1428] Setting key 18548a50e363b3f433adfea6165061a5
Sep 25 07:29:44 dist-test-slave-1drq krb5kdc[27969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321384, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.126@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:29:44.118568 28052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:44.110244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:45699 (local address 127.0.0.1:44128)
0925 07:29:44.110474 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:44.110508 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:44.110939 (+ 431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:44.111247 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:44.111249 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:44.111478 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:29:44.111625 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:44.111630 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:44.112666 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:44.112667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:44.113380 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:44.113383 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:44.113903 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:44.113904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:44.113960 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:44.114416 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:29:44.114422 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:29:44.115587 (+ 1165us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:29:44.117394 (+ 1807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:29:44.117399 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:29:44.117400 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:29:44.117555 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:29:44.117800 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:29:44.117800 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:29:44.117800 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:29:44.117864 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:29:44.118255 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:29:44.118258 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:29:44.118301 (+ 43us) client_negotiation.cc:769] Sending connection context
0925 07:29:44.118358 (+ 57us) client_negotiation.cc:240] Negotiation successful
0925 07:29:44.118448 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":109,"threads_started":1}
I20260925 07:29:44.118675 28053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:44.110323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44128 (local address 127.5.158.126:45699)
0925 07:29:44.110481 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:44.110488 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:44.110606 (+ 118us) server_negotiation.cc:407] Connection header received
0925 07:29:44.111101 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:44.111102 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:44.111125 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:44.111236 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:44.111783 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:44.112556 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:44.113461 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:44.113800 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:44.113921 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:44.115667 (+ 1746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:44.115674 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:44.115674 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:29:44.115681 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:44.117324 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:44.117636 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:44.117639 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:44.117639 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:44.117733 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:44.117927 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:44.117928 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:44.117928 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:44.118180 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:44.118271 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:44.118423 (+ 152us) server_negotiation.cc:299] Negotiation successful
0925 07:29:44.118519 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":128,"thread_start_us":61,"threads_started":1}
I20260925 07:29:44.119418 28050 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.158.125" port: 33185 } has no permanent_uuid. Determining permanent_uuid...
I20260925 07:29:44.119709 28052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:44.119607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.125:33185 (local address 127.0.0.1:59160)
0925 07:29:44.119649 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:44.119682 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.125:33185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260925 07:29:44.119872 28000 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.158.125:33185: connect: Connection refused (error 111)
I20260925 07:29:44.120308 28052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:44.120219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.125:33185 (local address 127.0.0.1:59174)
0925 07:29:44.120270 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:44.120293 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.125:33185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260925 07:29:44.120471 28050 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.158.125:33185: Network error: Client connection negotiation failed: client connection to 127.5.158.125:33185: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.5.158.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.125.keytab.
Entry for principal kudu/127.5.158.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.125.keytab.
I20260925 07:29:44.136195 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.125:33185
--webserver_interface=127.5.158.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.125.keytab
--principal=kudu/127.5.158.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:44165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.158.126:45699,127.5.158.125:33185,127.5.158.124:37331
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 07:29:44.169731 28050 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.158.125" port: 33185 } attempt: 1
I20260925 07:29:44.170266 28052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:44.170078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.125:33185 (local address 127.0.0.1:59176)
0925 07:29:44.170157 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:44.170222 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.125:33185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260925 07:29:44.170675 28052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:44.170551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.125:33185 (local address 127.0.0.1:59182)
0925 07:29:44.170606 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:44.170643 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.125:33185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260925 07:29:44.170796 28050 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.158.125:33185: Network error: Client connection negotiation failed: client connection to 127.5.158.125:33185: connect: Connection refused (error 111)
W20260925 07:29:44.214015 28057 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 07:29:44.214201 28057 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:29:44.214218 28057 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:29:44.214237 28057 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:29:44.215219 28057 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:29:44.215255 28057 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:29:44.215269 28057 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:29:44.215282 28057 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:29:44.215296 28057 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:29:44.216542 28057 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:44165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.158.126:45699,127.5.158.125:33185,127.5.158.124:37331
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.125.keytab
--principal=kudu/127.5.158.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.125:33185
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.158.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28057
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:29:44.216714 28057 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:29:44.216871 28057 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 07:29:44.221416 28057 server_base.cc:1034] running on GCE node
W20260925 07:29:44.221414 28062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:44.221432 28063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:44.221446 28065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:44.221941 28057 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:29:44.222249 28057 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:29:44.223402 28057 hybrid_clock.cc:648] HybridClock initialized: now 1790321384223377 us; error 43 us; skew 500 ppm
Sep 25 07:29:44 dist-test-slave-1drq krb5kdc[27969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321384, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:29:44.225665 28057 init.cc:377] Logged in from keytab as kudu/127.5.158.125@KRBTEST.COM (short username kudu)
I20260925 07:29:44.226069 28057 webserver.cc:460] Webserver started at http://127.5.158.125:46401/ using document root <none> and password file <none>
I20260925 07:29:44.226310 28057 fs_manager.cc:359] Metadata directory not provided
I20260925 07:29:44.226393 28057 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:29:44.226536 28057 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:29:44.227474 28057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/minicluster-data/master-1/data/instance:
uuid: "f4739757eb9d40928026a9b2ca721cbd"
format_stamp: "Formatted at 2026-09-25 07:29:44 on dist-test-slave-1drq"
server_key: "9dfeebd3d70d7c79d5d3a992ab89433b"
server_key_iv: "dcfae737f7c0f2b23709d0cb481be737"
server_key_version: "encryptionkey@0"
I20260925 07:29:44.227866 28057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/minicluster-data/master-1/wal/instance:
uuid: "f4739757eb9d40928026a9b2ca721cbd"
format_stamp: "Formatted at 2026-09-25 07:29:44 on dist-test-slave-1drq"
server_key: "9dfeebd3d70d7c79d5d3a992ab89433b"
server_key_iv: "dcfae737f7c0f2b23709d0cb481be737"
server_key_version: "encryptionkey@0"
I20260925 07:29:44.229202 28057 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 07:29:44.229887 28072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:44.230046 28057 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:29:44.230139 28057 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/minicluster-data/master-1/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/minicluster-data/master-1/wal
uuid: "f4739757eb9d40928026a9b2ca721cbd"
format_stamp: "Formatted at 2026-09-25 07:29:44 on dist-test-slave-1drq"
server_key: "9dfeebd3d70d7c79d5d3a992ab89433b"
server_key_iv: "dcfae737f7c0f2b23709d0cb481be737"
server_key_version: "encryptionkey@0"
I20260925 07:29:44.230242 28057 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:29:44.238932 28050 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.158.125" port: 33185 } attempt: 2
I20260925 07:29:44.239254 28052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:44.239138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.125:33185 (local address 127.0.0.1:59190)
0925 07:29:44.239191 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:44.239229 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.125:33185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260925 07:29:44.239548 28052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:44.239468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.125:33185 (local address 127.0.0.1:59206)
0925 07:29:44.239508 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:44.239528 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.125:33185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260925 07:29:44.239650 28050 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.158.125:33185: Network error: Client connection negotiation failed: client connection to 127.5.158.125:33185: connect: Connection refused (error 111)
I20260925 07:29:44.251094 28057 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:29:44.252846 28057 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:29:44.253001 28057 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:29:44.256886 28057 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.125:33185
I20260925 07:29:44.256924 28124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.125:33185 every 8 connection(s)
I20260925 07:29:44.257298 28057 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/minicluster-data/master-1/data/info.pb
I20260925 07:29:44.258136 28125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:29:44.259390 28125 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.158.126" port: 45699 } has no permanent_uuid. Determining permanent_uuid...
I20260925 07:29:44.260663 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 28057
I20260925 07:29:44.260780 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/minicluster-data/master-1/wal/instance
I20260925 07:29:44.260967 5753 external_mini_cluster.cc:1428] Setting key b7d4c1f9fd275653fff983b881a36911
Sep 25 07:29:44 dist-test-slave-1drq krb5kdc[27969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321384, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.125@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.158.124@KRBTEST.COM; defaulting to no policy
I20260925 07:29:44.269677 28053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:44.259953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44142 (local address 127.5.158.126:45699)
0925 07:29:44.260048 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:44.260050 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:44.260237 (+ 187us) server_negotiation.cc:407] Connection header received
0925 07:29:44.260922 (+ 685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:44.260923 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:44.260937 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:44.260984 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:44.261809 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:44.262284 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:44.263347 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:44.263660 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:44.263741 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:44.265895 (+ 2154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:44.265897 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:44.265898 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:29:44.265906 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:44.267995 (+ 2089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:44.268479 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:44.268481 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:44.268481 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:44.268549 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:44.268856 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:44.268857 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:44.268857 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:44.269019 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:44.269068 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:44.269473 (+ 405us) server_negotiation.cc:299] Negotiation successful
0925 07:29:44.269578 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260925 07:29:44.269765 28127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:44.259914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:45699 (local address 127.0.0.1:44142)
0925 07:29:44.260082 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:44.260121 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:44.260795 (+ 674us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:44.261053 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:44.261057 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:44.261425 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:29:44.261711 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:44.261716 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:44.262406 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:44.262407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:44.263229 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:44.263231 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:44.263776 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:44.263777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:44.263888 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:44.264363 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:29:44.264371 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:29:44.265799 (+ 1428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:29:44.268070 (+ 2271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:29:44.268076 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:29:44.268078 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:29:44.268369 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:29:44.268624 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:29:44.268625 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:29:44.268625 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:29:44.268777 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:29:44.269119 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:29:44.269125 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:29:44.269305 (+ 180us) client_negotiation.cc:769] Sending connection context
0925 07:29:44.269403 (+ 98us) client_negotiation.cc:240] Negotiation successful
0925 07:29:44.269564 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":74,"threads_started":1}
I20260925 07:29:44.270601 28125 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.158.125" port: 33185 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.124@KRBTEST.COM" created.
Sep 25 07:29:44 dist-test-slave-1drq krb5kdc[27969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321384, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.125@KRBTEST.COM for kudu/127.5.158.125@KRBTEST.COM
I20260925 07:29:44.278993 28127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:44.270856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.125:33185 (local address 127.0.0.1:59222)
0925 07:29:44.270905 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:44.270916 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:44.270957 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:44.271293 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:44.271294 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:44.271495 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:29:44.271554 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:44.271557 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:44.272905 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:44.272905 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:44.273753 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:44.273756 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:44.274244 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:44.274245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:44.274301 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:44.274736 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:29:44.274741 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:29:44.276204 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:29:44.277883 (+ 1679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:29:44.277884 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:29:44.277884 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:29:44.278094 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:29:44.278322 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:29:44.278323 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:29:44.278324 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:29:44.278392 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:29:44.278721 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:29:44.278721 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:29:44.278751 (+ 30us) client_negotiation.cc:769] Sending connection context
0925 07:29:44.278803 (+ 52us) client_negotiation.cc:240] Negotiation successful
0925 07:29:44.278875 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260925 07:29:44.279240 28129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:44.270938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59222 (local address 127.5.158.125:33185)
0925 07:29:44.271108 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:44.271113 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:44.271137 (+ 24us) server_negotiation.cc:407] Connection header received
0925 07:29:44.271161 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:44.271163 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:44.271178 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:44.271266 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:44.271676 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:44.272805 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:44.273854 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:44.274148 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:44.274244 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:44.276292 (+ 2048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:44.276298 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:44.276298 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:29:44.276304 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:44.277798 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:44.278178 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:44.278181 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:44.278181 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:44.278243 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:44.278462 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:44.278463 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:44.278464 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:44.278622 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:44.278690 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:44.279101 (+ 411us) server_negotiation.cc:299] Negotiation successful
0925 07:29:44.279161 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":76,"threads_started":1}
I20260925 07:29:44.280098 28125 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.158.124" port: 37331 } has no permanent_uuid. Determining permanent_uuid...
I20260925 07:29:44.280395 28127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:44.280301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.124:37331 (local address 127.0.0.1:48350)
0925 07:29:44.280332 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:44.280373 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.124:37331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260925 07:29:44.280540 28075 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.158.124:37331: connect: Connection refused (error 111)
I20260925 07:29:44.281013 28127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:44.280912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.124:37331 (local address 127.0.0.1:48364)
0925 07:29:44.280943 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:44.280973 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.124:37331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260925 07:29:44.281136 28125 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.158.124:37331: Network error: Client connection negotiation failed: client connection to 127.5.158.124:37331: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.124.keytab.
Entry for principal kudu/127.5.158.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.124.keytab.
I20260925 07:29:44.288380 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.124:37331
--webserver_interface=127.5.158.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.124.keytab
--principal=kudu/127.5.158.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:44165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.158.126:45699,127.5.158.125:33185,127.5.158.124:37331
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 07:29:44.330381 28125 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.158.124" port: 37331 } attempt: 1
I20260925 07:29:44.330821 28127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:44.330655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.124:37331 (local address 127.0.0.1:48376)
0925 07:29:44.330752 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:44.330789 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.124:37331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260925 07:29:44.330780 28050 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.158.125" port: 33185 } attempt: 3
I20260925 07:29:44.331607 28127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:44.331214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.124:37331 (local address 127.0.0.1:48386)
0925 07:29:44.331547 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:44.331570 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.124:37331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":317}
W20260925 07:29:44.331746 28125 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.158.124:37331: Network error: Client connection negotiation failed: client connection to 127.5.158.124:37331: connect: Connection refused (error 111)
Sep 25 07:29:44 dist-test-slave-1drq krb5kdc[27969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321384, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.126@KRBTEST.COM for kudu/127.5.158.125@KRBTEST.COM
I20260925 07:29:44.337729 28052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:44.331023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.125:33185 (local address 127.0.0.1:59226)
0925 07:29:44.331066 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:44.331077 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:44.331122 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:44.331287 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:44.331288 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:44.331422 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:29:44.331479 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:44.331481 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:44.332450 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:44.332451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:44.333054 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:44.333056 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:44.333546 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:44.333547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:44.333599 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:44.334058 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:29:44.334062 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:29:44.335336 (+ 1274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:29:44.336837 (+ 1501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:29:44.336837 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:29:44.336838 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:29:44.336981 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:29:44.337214 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:29:44.337215 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:29:44.337215 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:29:44.337262 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:29:44.337503 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:29:44.337503 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:29:44.337536 (+ 33us) client_negotiation.cc:769] Sending connection context
0925 07:29:44.337583 (+ 47us) client_negotiation.cc:240] Negotiation successful
0925 07:29:44.337667 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260925 07:29:44.337807 28129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:44.331077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59226 (local address 127.5.158.125:33185)
0925 07:29:44.331119 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:44.331121 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:44.331133 (+ 12us) server_negotiation.cc:407] Connection header received
0925 07:29:44.331180 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:44.331181 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:44.331191 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:44.331272 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:44.331552 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:44.332366 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:44.333164 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:44.333479 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:44.333541 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:44.335414 (+ 1873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:44.335415 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:44.335416 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:29:44.335422 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:44.336764 (+ 1342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:44.337069 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:44.337070 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:44.337070 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:44.337158 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:44.337319 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:44.337320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:44.337320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:44.337438 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:44.337501 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:44.337655 (+ 154us) server_negotiation.cc:299] Negotiation successful
0925 07:29:44.337728 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 07:29:44.338258 28050 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.158.124" port: 37331 } has no permanent_uuid. Determining permanent_uuid...
I20260925 07:29:44.338593 28052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:44.338472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.124:37331 (local address 127.0.0.1:48392)
0925 07:29:44.338521 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:44.338554 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.124:37331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260925 07:29:44.338866 28052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:44.338787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.124:37331 (local address 127.0.0.1:48398)
0925 07:29:44.338828 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:44.338848 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.124:37331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260925 07:29:44.338951 28050 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.158.124:37331: Network error: Client connection negotiation failed: client connection to 127.5.158.124:37331: connect: Connection refused (error 111)
W20260925 07:29:44.363852 28132 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 07:29:44.364049 28132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:29:44.364102 28132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:29:44.364140 28132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:29:44.365202 28132 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:29:44.365270 28132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:29:44.365293 28132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:29:44.365319 28132 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:29:44.365341 28132 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:29:44.366729 28132 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:44165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.158.126:45699,127.5.158.125:33185,127.5.158.124:37331
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.124.keytab
--principal=kudu/127.5.158.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.124:37331
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.158.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28132
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:29:44.366979 28132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:29:44.367192 28132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 07:29:44.370095 28050 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.158.124" port: 37331 } attempt: 1
I20260925 07:29:44.370481 28052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:44.370351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.124:37331 (local address 127.0.0.1:48408)
0925 07:29:44.370409 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:44.370446 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.124:37331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260925 07:29:44.370838 28052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:44.370755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.124:37331 (local address 127.0.0.1:48418)
0925 07:29:44.370786 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:44.370808 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.124:37331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260925 07:29:44.370945 28050 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.158.124:37331: Network error: Client connection negotiation failed: client connection to 127.5.158.124:37331: connect: Connection refused (error 111)
W20260925 07:29:44.371801 28138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:44.371790 28137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:44.371793 28140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:44.372063 28132 server_base.cc:1034] running on GCE node
I20260925 07:29:44.372298 28132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:29:44.372601 28132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:29:44.373741 28132 hybrid_clock.cc:648] HybridClock initialized: now 1790321384373718 us; error 34 us; skew 500 ppm
Sep 25 07:29:44 dist-test-slave-1drq krb5kdc[27969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321384, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:29:44.375878 28132 init.cc:377] Logged in from keytab as kudu/127.5.158.124@KRBTEST.COM (short username kudu)
I20260925 07:29:44.376292 28132 webserver.cc:460] Webserver started at http://127.5.158.124:43219/ using document root <none> and password file <none>
I20260925 07:29:44.376477 28132 fs_manager.cc:359] Metadata directory not provided
I20260925 07:29:44.376514 28132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:29:44.376592 28132 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:29:44.377502 28132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/minicluster-data/master-2/data/instance:
uuid: "5122974b702f4d56b99284342595ffc5"
format_stamp: "Formatted at 2026-09-25 07:29:44 on dist-test-slave-1drq"
server_key: "0cf875ad36d755c019f409274e3083ea"
server_key_iv: "9f0d863308cd3accdbe12c092dc9df91"
server_key_version: "encryptionkey@0"
I20260925 07:29:44.377749 28132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/minicluster-data/master-2/wal/instance:
uuid: "5122974b702f4d56b99284342595ffc5"
format_stamp: "Formatted at 2026-09-25 07:29:44 on dist-test-slave-1drq"
server_key: "0cf875ad36d755c019f409274e3083ea"
server_key_iv: "9f0d863308cd3accdbe12c092dc9df91"
server_key_version: "encryptionkey@0"
I20260925 07:29:44.378912 28132 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 07:29:44.379534 28147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:44.379801 28132 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 07:29:44.379894 28132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/minicluster-data/master-2/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/minicluster-data/master-2/wal
uuid: "5122974b702f4d56b99284342595ffc5"
format_stamp: "Formatted at 2026-09-25 07:29:44 on dist-test-slave-1drq"
server_key: "0cf875ad36d755c019f409274e3083ea"
server_key_iv: "9f0d863308cd3accdbe12c092dc9df91"
server_key_version: "encryptionkey@0"
I20260925 07:29:44.379951 28132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:29:44.398075 28132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:29:44.399703 28132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:29:44.399798 28132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:29:44.399911 28125 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.158.124" port: 37331 } attempt: 2
I20260925 07:29:44.400307 28127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:44.400161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.124:37331 (local address 127.0.0.1:48428)
0925 07:29:44.400233 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:44.400270 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.124:37331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260925 07:29:44.400758 28127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:44.400671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.124:37331 (local address 127.0.0.1:48430)
0925 07:29:44.400717 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:44.400741 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.124:37331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260925 07:29:44.400880 28125 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.158.124:37331: Network error: Client connection negotiation failed: client connection to 127.5.158.124:37331: connect: Connection refused (error 111)
I20260925 07:29:44.404220 28132 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.124:37331
I20260925 07:29:44.404264 28199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.124:37331 every 8 connection(s)
I20260925 07:29:44.404590 28132 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/minicluster-data/master-2/data/info.pb
I20260925 07:29:44.405596 28200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:29:44.406896 28200 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.158.126" port: 45699 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 07:29:44 dist-test-slave-1drq krb5kdc[27969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321384, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.124@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:29:44.412375 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 28132
I20260925 07:29:44.412505 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790321110323796-5753-0/minicluster-data/master-2/wal/instance
I20260925 07:29:44.412738 5753 external_mini_cluster.cc:1428] Setting key 26d25f871cfd7fea33de230d641aa9c0
I20260925 07:29:44.415068 28053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:44.407514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44144 (local address 127.5.158.126:45699)
0925 07:29:44.407573 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:44.407576 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:44.407770 (+ 194us) server_negotiation.cc:407] Connection header received
0925 07:29:44.408237 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:44.408239 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:44.408249 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:44.408302 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:44.408840 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:44.409389 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:44.410282 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:44.410466 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:44.410566 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:44.412413 (+ 1847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:44.412414 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:44.412415 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:29:44.412422 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:44.413836 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:44.414260 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:44.414261 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:44.414261 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:44.414317 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:44.414528 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:44.414529 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:44.414529 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:44.414642 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:44.414700 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:44.414913 (+ 213us) server_negotiation.cc:299] Negotiation successful
0925 07:29:44.414985 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 07:29:44.415154 28202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:44.407488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:45699 (local address 127.0.0.1:44144)
0925 07:29:44.407662 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:44.407691 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:44.408134 (+ 443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:44.408367 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:44.408370 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:44.408579 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:29:44.408741 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:44.408744 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:44.409484 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:44.409485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:44.410181 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:44.410183 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:44.410582 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:44.410583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:44.410707 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:44.411087 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:29:44.411094 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:29:44.412314 (+ 1220us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:29:44.413911 (+ 1597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:29:44.413918 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:29:44.413920 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:29:44.414166 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:29:44.414382 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:29:44.414382 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:29:44.414383 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:29:44.414465 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:29:44.414702 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:29:44.414706 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:29:44.414796 (+ 90us) client_negotiation.cc:769] Sending connection context
0925 07:29:44.414865 (+ 69us) client_negotiation.cc:240] Negotiation successful
0925 07:29:44.414958 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":78,"threads_started":1}
I20260925 07:29:44.415616 28200 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.158.125" port: 33185 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 07:29:44 dist-test-slave-1drq krb5kdc[27969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321383, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.158.124@KRBTEST.COM
Sep 25 07:29:44 dist-test-slave-1drq krb5kdc[27969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321384, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.124@KRBTEST.COM for kudu/127.5.158.125@KRBTEST.COM
I20260925 07:29:44.421574 28204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:44.413124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48438 (local address 127.5.158.124:37331)
0925 07:29:44.413319 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:44.413323 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:44.413364 (+ 41us) server_negotiation.cc:407] Connection header received
0925 07:29:44.413382 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:44.413393 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:44.413404 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:44.413468 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:44.414065 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:44.414942 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:44.415887 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:44.416173 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:44.416257 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:44.418591 (+ 2334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:44.418597 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:44.418597 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:29:44.418602 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:44.420267 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:44.420631 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:44.420634 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:44.420635 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:44.420693 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:44.420927 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:44.420927 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:44.420927 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:44.421077 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:44.421251 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:44.421383 (+ 132us) server_negotiation.cc:299] Negotiation successful
0925 07:29:44.421457 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":76,"threads_started":1}
I20260925 07:29:44.421908 28202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:44.415816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.125:33185 (local address 127.0.0.1:59232)
0925 07:29:44.415863 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:44.415871 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:44.415904 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:44.416007 (+ 103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:44.416007 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:44.416123 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:29:44.416162 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:44.416163 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:44.416733 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:44.416734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:44.417278 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:44.417280 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:44.417704 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:44.417705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:44.417767 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:44.418154 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:29:44.418158 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:29:44.419272 (+ 1114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:29:44.420775 (+ 1503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:29:44.420776 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:29:44.420776 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:29:44.420997 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:29:44.421228 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:29:44.421229 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:29:44.421229 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:29:44.421296 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:29:44.421588 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:29:44.421589 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:29:44.421645 (+ 56us) client_negotiation.cc:769] Sending connection context
0925 07:29:44.421742 (+ 97us) client_negotiation.cc:240] Negotiation successful
0925 07:29:44.421825 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260925 07:29:44.422204 28129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:44.415865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59232 (local address 127.5.158.125:33185)
0925 07:29:44.415914 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:44.415916 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:44.415924 (+ 8us) server_negotiation.cc:407] Connection header received
0925 07:29:44.415948 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:44.415948 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:44.415957 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:44.415996 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:44.416243 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:44.416657 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:44.417369 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:44.417601 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:44.417685 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:44.419370 (+ 1685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:44.419371 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:44.419372 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:29:44.419377 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:44.420684 (+ 1307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:44.421075 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:44.421076 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:44.421076 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:44.421160 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:44.421363 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:44.421364 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:44.421364 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:44.421487 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:44.421559 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:44.422049 (+ 490us) server_negotiation.cc:299] Negotiation successful
0925 07:29:44.422123 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260925 07:29:44.422720 28200 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.158.124" port: 37331 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 07:29:44 dist-test-slave-1drq krb5kdc[27969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321384, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.124@KRBTEST.COM for kudu/127.5.158.124@KRBTEST.COM
Sep 25 07:29:44 dist-test-slave-1drq krb5kdc[27969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321383, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:29:44.428689 28202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:44.422935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.124:37331 (local address 127.0.0.1:48444)
0925 07:29:44.422996 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:44.423005 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:44.423043 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:44.423158 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:44.423158 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:44.423284 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:29:44.423304 (+ 20us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:44.423305 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:44.423892 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:44.423893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:44.424330 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:44.424332 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:44.424697 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:44.424698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:44.424742 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:44.425035 (+ 293us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:29:44.425039 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:29:44.426163 (+ 1124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:29:44.427761 (+ 1598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:29:44.427762 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:29:44.427762 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:29:44.427888 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:29:44.428069 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:29:44.428069 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:29:44.428069 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:29:44.428109 (+ 40us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:29:44.428346 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:29:44.428346 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:29:44.428405 (+ 59us) client_negotiation.cc:769] Sending connection context
0925 07:29:44.428481 (+ 76us) client_negotiation.cc:240] Negotiation successful
0925 07:29:44.428583 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260925 07:29:44.428782 28204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:44.422995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48444 (local address 127.5.158.124:37331)
0925 07:29:44.423024 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:44.423026 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:44.423033 (+ 7us) server_negotiation.cc:407] Connection header received
0925 07:29:44.423095 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:44.423096 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:44.423104 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:44.423154 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:44.423388 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:44.423815 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:44.424421 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:44.424614 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:44.424704 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:44.426243 (+ 1539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:44.426244 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:44.426244 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:29:44.426249 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:44.427690 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:44.427958 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:44.427959 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:44.427959 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:44.428007 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:44.428167 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:44.428168 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:44.428168 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:44.428277 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:44.428325 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:44.428541 (+ 216us) server_negotiation.cc:299] Negotiation successful
0925 07:29:44.428635 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":16,"server-negotiator.queue_time_us":17}
I20260925 07:29:44.429407 28053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:44.422344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44156 (local address 127.5.158.126:45699)
0925 07:29:44.422375 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:44.422377 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:44.422385 (+ 8us) server_negotiation.cc:407] Connection header received
0925 07:29:44.422426 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:44.422427 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:44.422435 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:44.422482 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:44.423073 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:44.423486 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:44.424121 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:44.424316 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:44.424408 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:44.426800 (+ 2392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:44.426801 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:44.426802 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:29:44.426808 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:44.428154 (+ 1346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:44.428529 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:44.428530 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:44.428530 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:44.428582 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:44.428816 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:44.428817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:44.428817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:44.428937 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:44.429020 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:44.429258 (+ 238us) server_negotiation.cc:299] Negotiation successful
0925 07:29:44.429328 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260925 07:29:44.429932 28200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5122974b702f4d56b99284342595ffc5: Bootstrap starting.
I20260925 07:29:44.430569 28200 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5122974b702f4d56b99284342595ffc5: Neither blocks nor log segments found. Creating new log.
I20260925 07:29:44.430774 28200 log.cc:824] T 00000000000000000000000000000000 P 5122974b702f4d56b99284342595ffc5: Log is configured to *not* fsync() on all Append() calls
I20260925 07:29:44.431360 28200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5122974b702f4d56b99284342595ffc5: No bootstrap required, opened a new log
I20260925 07:29:44.433135 28200 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5122974b702f4d56b99284342595ffc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08be0716dc1741b3b8ea2b59242728b8" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 45699 } } peers { permanent_uuid: "f4739757eb9d40928026a9b2ca721cbd" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 33185 } } peers { permanent_uuid: "5122974b702f4d56b99284342595ffc5" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 37331 } }
I20260925 07:29:44.433251 28200 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5122974b702f4d56b99284342595ffc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5122974b702f4d56b99284342595ffc5, State: Initialized, Role: FOLLOWER
I20260925 07:29:44.433329 28200 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5122974b702f4d56b99284342595ffc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08be0716dc1741b3b8ea2b59242728b8" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 45699 } } peers { permanent_uuid: "f4739757eb9d40928026a9b2ca721cbd" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 33185 } } peers { permanent_uuid: "5122974b702f4d56b99284342595ffc5" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 37331 } }
I20260925 07:29:44.433591 28200 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5122974b702f4d56b99284342595ffc5 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:29:44.433579 28206 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5122974b702f4d56b99284342595ffc5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08be0716dc1741b3b8ea2b59242728b8" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 45699 } } peers { permanent_uuid: "f4739757eb9d40928026a9b2ca721cbd" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 33185 } } peers { permanent_uuid: "5122974b702f4d56b99284342595ffc5" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 37331 } } }
I20260925 07:29:44.433694 28206 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5122974b702f4d56b99284342595ffc5 [sys.catalog]: This master's current role is: FOLLOWER
W20260925 07:29:44.435448 28217 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5122974b702f4d56b99284342595ffc5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 07:29:44.435539 28217 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Sep 25 07:29:44 dist-test-slave-1drq krb5kdc[27969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321383, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.158.125@KRBTEST.COM
I20260925 07:29:44.435714 28132 master_runner.cc:186] Error getting master registration for 127.5.158.126:45699: OK, instance_id { permanent_uuid: "08be0716dc1741b3b8ea2b59242728b8" instance_seqno: 1790321384105973 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 07:29:44.438131 28132 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 07:29:44.438925 28129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:44.431388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59248 (local address 127.5.158.125:33185)
0925 07:29:44.431422 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:44.431424 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:44.431449 (+ 25us) server_negotiation.cc:407] Connection header received
0925 07:29:44.431468 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:44.431469 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:44.431477 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:44.431518 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:44.431930 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:44.432638 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:44.433374 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:44.433631 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:44.433694 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:44.436174 (+ 2480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:44.436176 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:44.436176 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:29:44.436183 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:44.437832 (+ 1649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:44.438206 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:44.438207 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:44.438207 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:44.438261 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:44.438457 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:44.438458 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:44.438458 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:44.438548 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:44.438727 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:44.438759 (+ 32us) server_negotiation.cc:299] Negotiation successful
0925 07:29:44.438839 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 07:29:44.440196 5753 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 07:29:44.441357 28024 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "08be0716dc1741b3b8ea2b59242728b8"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44156
I20260925 07:29:44.446096 28050 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.158.124" port: 37331 } attempt: 2
Sep 25 07:29:44 dist-test-slave-1drq krb5kdc[27969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321384, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.126@KRBTEST.COM for kudu/127.5.158.124@KRBTEST.COM
I20260925 07:29:44.452030 28204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:44.446365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48458 (local address 127.5.158.124:37331)
0925 07:29:44.446416 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:44.446418 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:44.446424 (+ 6us) server_negotiation.cc:407] Connection header received
0925 07:29:44.446438 (+ 14us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:44.446439 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:44.446446 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:44.446507 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:44.446759 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:44.447158 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:44.447847 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:44.448008 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:44.448086 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:44.449629 (+ 1543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:44.449630 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:44.449630 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:29:44.449636 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:44.450931 (+ 1295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:44.451263 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:44.451264 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:44.451264 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:44.451308 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:44.451491 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:44.451492 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:44.451492 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:44.451619 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:44.451731 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:44.451884 (+ 153us) server_negotiation.cc:299] Negotiation successful
0925 07:29:44.451947 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 07:29:44.452035 28052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:44.446297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.124:37331 (local address 127.0.0.1:48458)
0925 07:29:44.446343 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:44.446352 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:44.446384 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:44.446510 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:44.446511 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:44.446631 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:29:44.446657 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:44.446658 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:44.447248 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:44.447249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:44.447756 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:44.447757 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:44.448087 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:44.448088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:44.448139 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:44.448496 (+ 357us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:29:44.448500 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:29:44.449552 (+ 1052us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:29:44.451030 (+ 1478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:29:44.451030 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:29:44.451031 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:29:44.451191 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:29:44.451377 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:29:44.451378 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:29:44.451378 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:29:44.451425 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:29:44.451727 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:29:44.451728 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:29:44.451769 (+ 41us) client_negotiation.cc:769] Sending connection context
0925 07:29:44.451860 (+ 91us) client_negotiation.cc:240] Negotiation successful
0925 07:29:44.451946 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260925 07:29:44.453627 28050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08be0716dc1741b3b8ea2b59242728b8: Bootstrap starting.
I20260925 07:29:44.454255 28050 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 08be0716dc1741b3b8ea2b59242728b8: Neither blocks nor log segments found. Creating new log.
I20260925 07:29:44.454525 28050 log.cc:824] T 00000000000000000000000000000000 P 08be0716dc1741b3b8ea2b59242728b8: Log is configured to *not* fsync() on all Append() calls
I20260925 07:29:44.455263 28050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08be0716dc1741b3b8ea2b59242728b8: No bootstrap required, opened a new log
I20260925 07:29:44.456454 28050 raft_consensus.cc:348] T 00000000000000000000000000000000 P 08be0716dc1741b3b8ea2b59242728b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08be0716dc1741b3b8ea2b59242728b8" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 45699 } } peers { permanent_uuid: "f4739757eb9d40928026a9b2ca721cbd" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 33185 } } peers { permanent_uuid: "5122974b702f4d56b99284342595ffc5" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 37331 } }
I20260925 07:29:44.456574 28050 raft_consensus.cc:729] T 00000000000000000000000000000000 P 08be0716dc1741b3b8ea2b59242728b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08be0716dc1741b3b8ea2b59242728b8, State: Initialized, Role: FOLLOWER
I20260925 07:29:44.456643 28050 consensus_queue.cc:260] T 00000000000000000000000000000000 P 08be0716dc1741b3b8ea2b59242728b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08be0716dc1741b3b8ea2b59242728b8" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 45699 } } peers { permanent_uuid: "f4739757eb9d40928026a9b2ca721cbd" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 33185 } } peers { permanent_uuid: "5122974b702f4d56b99284342595ffc5" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 37331 } }
I20260925 07:29:44.456869 28220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08be0716dc1741b3b8ea2b59242728b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08be0716dc1741b3b8ea2b59242728b8" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 45699 } } peers { permanent_uuid: "f4739757eb9d40928026a9b2ca721cbd" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 33185 } } peers { permanent_uuid: "5122974b702f4d56b99284342595ffc5" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 37331 } } }
I20260925 07:29:44.456928 28024 raft_consensus.cc:482] T 00000000000000000000000000000000 P 08be0716dc1741b3b8ea2b59242728b8 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260925 07:29:44.456913 28050 sys_catalog.cc:564] T 00000000000000000000000000000000 P 08be0716dc1741b3b8ea2b59242728b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:29:44.456964 28220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08be0716dc1741b3b8ea2b59242728b8 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 07:29:44.457010 28024 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 08be0716dc1741b3b8ea2b59242728b8 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:29:44.457708 28024 raft_consensus.cc:504] T 00000000000000000000000000000000 P 08be0716dc1741b3b8ea2b59242728b8 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08be0716dc1741b3b8ea2b59242728b8" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 45699 } } peers { permanent_uuid: "f4739757eb9d40928026a9b2ca721cbd" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 33185 } } peers { permanent_uuid: "5122974b702f4d56b99284342595ffc5" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 37331 } }
I20260925 07:29:44.457938 28024 leader_election.cc:288] T 00000000000000000000000000000000 P 08be0716dc1741b3b8ea2b59242728b8 [CANDIDATE]: Term 1 election: Requested vote from peers f4739757eb9d40928026a9b2ca721cbd (127.5.158.125:33185), 5122974b702f4d56b99284342595ffc5 (127.5.158.124:37331)
I20260925 07:29:44.458420 28174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "08be0716dc1741b3b8ea2b59242728b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "5122974b702f4d56b99284342595ffc5"
I20260925 07:29:44.458549 28174 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5122974b702f4d56b99284342595ffc5 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:29:44.459482 28174 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5122974b702f4d56b99284342595ffc5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08be0716dc1741b3b8ea2b59242728b8 in term 1.
I20260925 07:29:44.459218 27982 master_runner.cc:186] Error getting master registration for 127.5.158.125:33185: OK, instance_id { permanent_uuid: "f4739757eb9d40928026a9b2ca721cbd" instance_seqno: 1790321384255822 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 07:29:44.461725 27982 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 07:29:44.461812 28000 leader_election.cc:302] T 00000000000000000000000000000000 P 08be0716dc1741b3b8ea2b59242728b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08be0716dc1741b3b8ea2b59242728b8, 5122974b702f4d56b99284342595ffc5; no voters:
I20260925 07:29:44.461923 28220 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 08be0716dc1741b3b8ea2b59242728b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:29:44.462059 28220 raft_consensus.cc:686] T 00000000000000000000000000000000 P 08be0716dc1741b3b8ea2b59242728b8 [term 1 LEADER]: Becoming Leader. State: Replica: 08be0716dc1741b3b8ea2b59242728b8, State: Running, Role: LEADER
I20260925 07:29:44.462143 28220 consensus_queue.cc:237] T 00000000000000000000000000000000 P 08be0716dc1741b3b8ea2b59242728b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08be0716dc1741b3b8ea2b59242728b8" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 45699 } } peers { permanent_uuid: "f4739757eb9d40928026a9b2ca721cbd" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 33185 } } peers { permanent_uuid: "5122974b702f4d56b99284342595ffc5" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 37331 } }
I20260925 07:29:44.462495 28220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08be0716dc1741b3b8ea2b59242728b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 08be0716dc1741b3b8ea2b59242728b8. Latest consensus state: current_term: 1 leader_uuid: "08be0716dc1741b3b8ea2b59242728b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08be0716dc1741b3b8ea2b59242728b8" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 45699 } } peers { permanent_uuid: "f4739757eb9d40928026a9b2ca721cbd" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 33185 } } peers { permanent_uuid: "5122974b702f4d56b99284342595ffc5" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 37331 } } }
I20260925 07:29:44.462581 28220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08be0716dc1741b3b8ea2b59242728b8 [sys.catalog]: This master's current role is: LEADER
I20260925 07:29:44.462801 28241 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 07:29:44.463207 28241 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 07:29:44.464959 28174 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5122974b702f4d56b99284342595ffc5 [term 1 FOLLOWER]: Refusing update from remote peer 08be0716dc1741b3b8ea2b59242728b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:29:44.465368 28220 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 08be0716dc1741b3b8ea2b59242728b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5122974b702f4d56b99284342595ffc5" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 37331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:29:44.466476 28206 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5122974b702f4d56b99284342595ffc5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 08be0716dc1741b3b8ea2b59242728b8. Latest consensus state: current_term: 1 leader_uuid: "08be0716dc1741b3b8ea2b59242728b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08be0716dc1741b3b8ea2b59242728b8" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 45699 } } peers { permanent_uuid: "f4739757eb9d40928026a9b2ca721cbd" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 33185 } } peers { permanent_uuid: "5122974b702f4d56b99284342595ffc5" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 37331 } } }
I20260925 07:29:44.466548 28206 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5122974b702f4d56b99284342595ffc5 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 07:29:44.470371 28220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08be0716dc1741b3b8ea2b59242728b8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "08be0716dc1741b3b8ea2b59242728b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08be0716dc1741b3b8ea2b59242728b8" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 45699 } } peers { permanent_uuid: "f4739757eb9d40928026a9b2ca721cbd" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 33185 } } peers { permanent_uuid: "5122974b702f4d56b99284342595ffc5" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 37331 } } }
I20260925 07:29:44.470463 28220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08be0716dc1741b3b8ea2b59242728b8 [sys.catalog]: This master's current role is: LEADER
I20260925 07:29:44.470872 28206 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5122974b702f4d56b99284342595ffc5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "08be0716dc1741b3b8ea2b59242728b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08be0716dc1741b3b8ea2b59242728b8" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 45699 } } peers { permanent_uuid: "f4739757eb9d40928026a9b2ca721cbd" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 33185 } } peers { permanent_uuid: "5122974b702f4d56b99284342595ffc5" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 37331 } } }
I20260925 07:29:44.470978 28206 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5122974b702f4d56b99284342595ffc5 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 07:29:44.471024 28244 mvcc.cc:205] Tried to move back new op lower bound from 7333156390765830144 to 7333156390757519360. Current Snapshot: MvccSnapshot[applied={T|T < 7333156390765830144}]
I20260925 07:29:44.471621 28053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:44.459086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44158 (local address 127.5.158.126:45699)
0925 07:29:44.459115 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:44.459117 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:44.459636 (+ 519us) server_negotiation.cc:407] Connection header received
0925 07:29:44.459729 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:44.459730 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:44.459738 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:44.459761 (+ 23us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:44.460378 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:44.461170 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:44.463079 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:44.463337 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:44.463397 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:44.465761 (+ 2364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:44.465762 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:44.465762 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:29:44.465769 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:44.467776 (+ 2007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:44.470204 (+ 2428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:44.470206 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:44.470206 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:44.470262 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:44.470725 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:44.470726 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:44.470726 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:44.470873 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:44.470954 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:44.471443 (+ 489us) server_negotiation.cc:299] Negotiation successful
0925 07:29:44.471514 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260925 07:29:44.472529 28204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:44.459407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48462 (local address 127.5.158.124:37331)
0925 07:29:44.459434 (+ 27us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:44.459436 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:44.463990 (+ 4554us) server_negotiation.cc:407] Connection header received
0925 07:29:44.464072 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:44.464073 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:44.464081 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:44.464105 (+ 24us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:44.464774 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:44.465463 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:44.466456 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:44.466736 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:44.466773 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:44.468099 (+ 1326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:44.468100 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:44.468101 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:29:44.468107 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:44.470043 (+ 1936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:44.471323 (+ 1280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:44.471324 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:44.471324 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:44.471374 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:44.471890 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:44.471891 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:44.471891 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:44.472006 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:44.472073 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:44.472371 (+ 298us) server_negotiation.cc:299] Negotiation successful
0925 07:29:44.472450 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260925 07:29:44.473433 28129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:44.459169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59252 (local address 127.5.158.125:33185)
0925 07:29:44.459205 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:44.459206 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:44.460448 (+ 1242us) server_negotiation.cc:407] Connection header received
0925 07:29:44.460580 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:44.460581 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:44.460591 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:44.460618 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:44.461333 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:44.462022 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:44.462989 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:44.463648 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:44.463684 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:44.469086 (+ 5402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:44.469087 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:44.469088 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:29:44.469094 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:44.470730 (+ 1636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:44.471570 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:44.471572 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:44.471572 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:44.471644 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:44.472674 (+ 1030us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:44.472676 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:44.472676 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:44.472827 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:44.472903 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:44.473258 (+ 355us) server_negotiation.cc:299] Negotiation successful
0925 07:29:44.473340 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 07:29:44.477696 28241 catalog_manager.cc:1347] Generated new cluster ID: e5b4c92d93454fbaafe532bbd1c2c7fd
I20260925 07:29:44.477741 28241 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 07:29:44.487677 28241 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 07:29:44.488444 28241 catalog_manager.cc:1504] Loading token signing keys...
I20260925 07:29:44.492017 28125 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.158.124" port: 37331 } attempt: 3
I20260925 07:29:44.493882 28241 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 08be0716dc1741b3b8ea2b59242728b8: Generated new TSK 0
I20260925 07:29:44.494127 28241 catalog_manager.cc:1514] Initializing in-progress tserver states...
Sep 25 07:29:44 dist-test-slave-1drq krb5kdc[27969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321384, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.125@KRBTEST.COM for kudu/127.5.158.124@KRBTEST.COM
I20260925 07:29:44.500912 28204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:44.492490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48470 (local address 127.5.158.124:37331)
0925 07:29:44.492536 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:44.492538 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:44.492545 (+ 7us) server_negotiation.cc:407] Connection header received
0925 07:29:44.492564 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:44.492564 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:44.492573 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:44.492618 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:44.493013 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:44.493603 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:44.494391 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:44.494769 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:44.494862 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:44.496797 (+ 1935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:44.496798 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:44.496798 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:29:44.496805 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:44.498439 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:44.499483 (+ 1044us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:44.499484 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:44.499485 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:44.499545 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:44.499776 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:44.499777 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:44.499778 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:44.499973 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:44.500321 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:44.500499 (+ 178us) server_negotiation.cc:299] Negotiation successful
0925 07:29:44.500683 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 07:29:44.501207 28127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:44.492257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.124:37331 (local address 127.0.0.1:48470)
0925 07:29:44.492366 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:44.492378 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:44.492420 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:44.492660 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:44.492662 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:44.492889 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:29:44.492934 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:44.492936 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:44.493674 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:44.493675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:44.494329 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:44.494331 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:44.494862 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:44.494863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:44.494919 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:44.495348 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:29:44.495353 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:29:44.496718 (+ 1365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:29:44.498538 (+ 1820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:29:44.498539 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:29:44.498539 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:29:44.498764 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:29:44.499615 (+ 851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:29:44.499616 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:29:44.499616 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:29:44.499694 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:29:44.500092 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:29:44.500093 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:29:44.500151 (+ 58us) client_negotiation.cc:769] Sending connection context
0925 07:29:44.500220 (+ 69us) client_negotiation.cc:240] Negotiation successful
0925 07:29:44.500562 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91}
I20260925 07:29:44.502899 28125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4739757eb9d40928026a9b2ca721cbd: Bootstrap starting.
I20260925 07:29:44.503608 28125 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f4739757eb9d40928026a9b2ca721cbd: Neither blocks nor log segments found. Creating new log.
I20260925 07:29:44.503906 28125 log.cc:824] T 00000000000000000000000000000000 P f4739757eb9d40928026a9b2ca721cbd: Log is configured to *not* fsync() on all Append() calls
I20260925 07:29:44.508721 28125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4739757eb9d40928026a9b2ca721cbd: No bootstrap required, opened a new log
I20260925 07:29:44.510694 28125 raft_consensus.cc:348] T 00000000000000000000000000000000 P f4739757eb9d40928026a9b2ca721cbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08be0716dc1741b3b8ea2b59242728b8" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 45699 } } peers { permanent_uuid: "f4739757eb9d40928026a9b2ca721cbd" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 33185 } } peers { permanent_uuid: "5122974b702f4d56b99284342595ffc5" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 37331 } }
I20260925 07:29:44.510807 28125 raft_consensus.cc:729] T 00000000000000000000000000000000 P f4739757eb9d40928026a9b2ca721cbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4739757eb9d40928026a9b2ca721cbd, State: Initialized, Role: FOLLOWER
I20260925 07:29:44.510882 28125 consensus_queue.cc:260] T 00000000000000000000000000000000 P f4739757eb9d40928026a9b2ca721cbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08be0716dc1741b3b8ea2b59242728b8" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 45699 } } peers { permanent_uuid: "f4739757eb9d40928026a9b2ca721cbd" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 33185 } } peers { permanent_uuid: "5122974b702f4d56b99284342595ffc5" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 37331 } }
I20260925 07:29:44.511154 28125 sys_catalog.cc:564] T 00000000000000000000000000000000 P f4739757eb9d40928026a9b2ca721cbd [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:29:44.511240 28099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "08be0716dc1741b3b8ea2b59242728b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f4739757eb9d40928026a9b2ca721cbd"
I20260925 07:29:44.511327 28099 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f4739757eb9d40928026a9b2ca721cbd [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:29:44.512197 28099 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f4739757eb9d40928026a9b2ca721cbd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08be0716dc1741b3b8ea2b59242728b8 in term 1.
I20260925 07:29:44.511104 28251 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4739757eb9d40928026a9b2ca721cbd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08be0716dc1741b3b8ea2b59242728b8" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 45699 } } peers { permanent_uuid: "f4739757eb9d40928026a9b2ca721cbd" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 33185 } } peers { permanent_uuid: "5122974b702f4d56b99284342595ffc5" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 37331 } } }
I20260925 07:29:44.512789 28251 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4739757eb9d40928026a9b2ca721cbd [sys.catalog]: This master's current role is: FOLLOWER
I20260925 07:29:44.512393 28098 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f4739757eb9d40928026a9b2ca721cbd [term 1 FOLLOWER]: Refusing update from remote peer 08be0716dc1741b3b8ea2b59242728b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:29:44.513190 28247 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 08be0716dc1741b3b8ea2b59242728b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4739757eb9d40928026a9b2ca721cbd" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 07:29:44.514026 28262 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f4739757eb9d40928026a9b2ca721cbd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 07:29:44.514096 28262 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 07:29:44.515197 28251 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4739757eb9d40928026a9b2ca721cbd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 08be0716dc1741b3b8ea2b59242728b8. Latest consensus state: current_term: 1 leader_uuid: "08be0716dc1741b3b8ea2b59242728b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08be0716dc1741b3b8ea2b59242728b8" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 45699 } } peers { permanent_uuid: "f4739757eb9d40928026a9b2ca721cbd" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 33185 } } peers { permanent_uuid: "5122974b702f4d56b99284342595ffc5" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 37331 } } }
I20260925 07:29:44.515272 28251 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4739757eb9d40928026a9b2ca721cbd [sys.catalog]: This master's current role is: FOLLOWER
I20260925 07:29:44.515455 28251 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4739757eb9d40928026a9b2ca721cbd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "08be0716dc1741b3b8ea2b59242728b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08be0716dc1741b3b8ea2b59242728b8" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 45699 } } peers { permanent_uuid: "f4739757eb9d40928026a9b2ca721cbd" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 33185 } } peers { permanent_uuid: "5122974b702f4d56b99284342595ffc5" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 37331 } } }
I20260925 07:29:44.515504 28251 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4739757eb9d40928026a9b2ca721cbd [sys.catalog]: This master's current role is: FOLLOWER
W20260925 07:29:44.531317 28053 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:29:44.529906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44170 (local address 127.5.158.126:45699)
0925 07:29:44.529942 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:44.529945 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:44.530727 (+ 782us) server_negotiation.cc:407] Connection header received
0925 07:29:44.530818 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:44.530819 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:44.530830 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:44.530857 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:44.531243 (+ 386us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44170: BlockingRecv error: recv got EOF from 127.0.0.1:44170 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260925 07:29:44.532107 28129 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:29:44.530237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59260 (local address 127.5.158.125:33185)
0925 07:29:44.530284 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:44.530286 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:44.531520 (+ 1234us) server_negotiation.cc:407] Connection header received
0925 07:29:44.531727 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:44.531728 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:44.531738 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:44.531771 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:44.532054 (+ 283us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59260: BlockingRecv error: recv got EOF from 127.0.0.1:59260 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
W20260925 07:29:44.532714 28204 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:29:44.530497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48472 (local address 127.5.158.124:37331)
0925 07:29:44.530528 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:44.530529 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:44.531679 (+ 1150us) server_negotiation.cc:407] Connection header received
0925 07:29:44.532300 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:44.532302 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:44.532311 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:44.532339 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:44.532656 (+ 317us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48472: BlockingRecv error: recv got EOF from 127.0.0.1:48472 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260925 07:29:44.535029 28129 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:29:44.534329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59264 (local address 127.5.158.125:33185)
0925 07:29:44.534359 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:44.534361 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:44.534546 (+ 185us) server_negotiation.cc:407] Connection header received
0925 07:29:44.534649 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:44.534649 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:44.534663 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:44.534704 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:44.534994 (+ 290us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59264: BlockingRecv error: recv got EOF from 127.0.0.1:59264 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260925 07:29:44.537679 28129 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:29:44.537000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59274 (local address 127.5.158.125:33185)
0925 07:29:44.537026 (+ 26us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:44.537027 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:44.537262 (+ 235us) server_negotiation.cc:407] Connection header received
0925 07:29:44.537338 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:44.537338 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:44.537345 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:44.537395 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:44.537645 (+ 250us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59274: BlockingRecv error: recv got EOF from 127.0.0.1:59274 (error 108)
Metrics: {"server-negotiator.queue_time_us":14}
W20260925 07:29:44.541342 28129 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:29:44.540627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59276 (local address 127.5.158.125:33185)
0925 07:29:44.540656 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:44.540658 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:44.540865 (+ 207us) server_negotiation.cc:407] Connection header received
0925 07:29:44.540952 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:44.540953 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:44.540961 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:44.540988 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:44.541305 (+ 317us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59276: BlockingRecv error: recv got EOF from 127.0.0.1:59276 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260925 07:29:44.546994 28129 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:29:44.546314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59288 (local address 127.5.158.125:33185)
0925 07:29:44.546341 (+ 27us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:44.546342 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:44.546541 (+ 199us) server_negotiation.cc:407] Connection header received
0925 07:29:44.546620 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:44.546620 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:44.546628 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:44.546653 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:44.546961 (+ 308us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59288: BlockingRecv error: recv got EOF from 127.0.0.1:59288 (error 108)
Metrics: {"server-negotiator.queue_time_us":14}
W20260925 07:29:44.556602 28129 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:29:44.555920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59296 (local address 127.5.158.125:33185)
0925 07:29:44.555947 (+ 27us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:44.555949 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:44.556160 (+ 211us) server_negotiation.cc:407] Connection header received
0925 07:29:44.556241 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:44.556241 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:44.556249 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:44.556284 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:44.556570 (+ 286us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59296: BlockingRecv error: recv got EOF from 127.0.0.1:59296 (error 108)
Metrics: {"server-negotiator.queue_time_us":14}
W20260925 07:29:44.574498 28129 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:29:44.573689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59312 (local address 127.5.158.125:33185)
0925 07:29:44.573730 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:44.573732 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:44.573994 (+ 262us) server_negotiation.cc:407] Connection header received
0925 07:29:44.574088 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:44.574089 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:44.574099 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:44.574128 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:44.574454 (+ 326us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59312: BlockingRecv error: recv got EOF from 127.0.0.1:59312 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260925 07:29:44.608568 28129 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:29:44.607676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59320 (local address 127.5.158.125:33185)
0925 07:29:44.607718 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:44.607720 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:44.608002 (+ 282us) server_negotiation.cc:407] Connection header received
0925 07:29:44.608107 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:44.608108 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:44.608120 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:44.608151 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:44.608515 (+ 364us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59320: BlockingRecv error: recv got EOF from 127.0.0.1:59320 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260925 07:29:44.674510 28129 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:29:44.673795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59332 (local address 127.5.158.125:33185)
0925 07:29:44.673864 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:44.673867 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:44.673970 (+ 103us) server_negotiation.cc:407] Connection header received
0925 07:29:44.674011 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:44.674012 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:44.674022 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:44.674072 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:44.674436 (+ 364us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59332: BlockingRecv error: recv got EOF from 127.0.0.1:59332 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260925 07:29:44.804283 28129 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:29:44.803556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59344 (local address 127.5.158.125:33185)
0925 07:29:44.803625 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:44.803634 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:44.803728 (+ 94us) server_negotiation.cc:407] Connection header received
0925 07:29:44.803828 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:44.803829 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:44.803840 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:44.803904 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:44.804223 (+ 319us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59344: BlockingRecv error: recv got EOF from 127.0.0.1:59344 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
I20260925 07:29:44.941893 28220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08be0716dc1741b3b8ea2b59242728b8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "08be0716dc1741b3b8ea2b59242728b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08be0716dc1741b3b8ea2b59242728b8" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 45699 } } peers { permanent_uuid: "f4739757eb9d40928026a9b2ca721cbd" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 33185 } } peers { permanent_uuid: "5122974b702f4d56b99284342595ffc5" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 37331 } } }
I20260925 07:29:44.942058 28220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08be0716dc1741b3b8ea2b59242728b8 [sys.catalog]: This master's current role is: LEADER
W20260925 07:29:45.062335 28129 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:29:45.061697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59360 (local address 127.5.158.125:33185)
0925 07:29:45.061783 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:45.061785 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:45.061796 (+ 11us) server_negotiation.cc:407] Connection header received
0925 07:29:45.061855 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:45.061856 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:45.061867 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:45.061943 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:45.062270 (+ 327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59360: BlockingRecv error: recv got EOF from 127.0.0.1:59360 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 07:29:45.435989 28217 catalog_manager.cc:1259] Loaded cluster ID: e5b4c92d93454fbaafe532bbd1c2c7fd
I20260925 07:29:45.436046 28217 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5122974b702f4d56b99284342595ffc5: loading cluster ID for follower catalog manager: success
I20260925 07:29:45.436731 28217 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5122974b702f4d56b99284342595ffc5: acquiring CA information for follower catalog manager: success
I20260925 07:29:45.436950 28217 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5122974b702f4d56b99284342595ffc5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 07:29:45.514515 28262 catalog_manager.cc:1259] Loaded cluster ID: e5b4c92d93454fbaafe532bbd1c2c7fd
I20260925 07:29:45.514585 28262 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f4739757eb9d40928026a9b2ca721cbd: loading cluster ID for follower catalog manager: success
I20260925 07:29:45.515429 28262 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f4739757eb9d40928026a9b2ca721cbd: acquiring CA information for follower catalog manager: success
I20260925 07:29:45.515735 28262 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f4739757eb9d40928026a9b2ca721cbd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 07:29:45.578244 28333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:45.575697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59366 (local address 127.5.158.125:33185)
0925 07:29:45.575808 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:45.575810 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:45.575904 (+ 94us) server_negotiation.cc:407] Connection header received
0925 07:29:45.575944 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:45.575945 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:45.575988 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:45.576082 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 07:29:45.576202 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:45.576654 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:45.577469 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:45.577654 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:45.577706 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:45.577895 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 07:29:45.578037 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 07:29:45.578106 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:45.578165 (+ 59us) server_negotiation.cc:299] Negotiation successful
0925 07:29:45.578182 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":66,"threads_started":1}
I20260925 07:29:45.582342 28339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:45.580112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48484 (local address 127.5.158.124:37331)
0925 07:29:45.580198 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:45.580200 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:45.580206 (+ 6us) server_negotiation.cc:407] Connection header received
0925 07:29:45.580246 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:45.580247 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:45.580277 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:45.580341 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 07:29:45.580424 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:45.580895 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:45.581618 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:45.581816 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:45.581868 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:45.581991 (+ 123us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 07:29:45.582075 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 07:29:45.582123 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:45.582195 (+ 72us) server_negotiation.cc:299] Negotiation successful
0925 07:29:45.582210 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71,"thread_start_us":45,"threads_started":1}
I20260925 07:29:45.586720 28333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:45.584141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59382 (local address 127.5.158.125:33185)
0925 07:29:45.584174 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:45.584176 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:45.584321 (+ 145us) server_negotiation.cc:407] Connection header received
0925 07:29:45.584415 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:45.584416 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:45.584427 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:45.584455 (+ 28us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 07:29:45.584623 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:45.585052 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:45.586026 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:45.586295 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:45.586354 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:45.586489 (+ 135us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 07:29:45.586530 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 07:29:45.586575 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:45.586645 (+ 70us) server_negotiation.cc:299] Negotiation successful
0925 07:29:45.586663 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260925 07:29:45.587153 28345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:45.584025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44182 (local address 127.5.158.126:45699)
0925 07:29:45.584155 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:45.584157 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:45.584166 (+ 9us) server_negotiation.cc:407] Connection header received
0925 07:29:45.584210 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:45.584211 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:45.584259 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:45.584474 (+ 215us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 07:29:45.584514 (+ 40us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:45.585055 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:45.586192 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:45.586538 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:45.586594 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:45.586769 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 07:29:45.586857 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 07:29:45.587037 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:45.587054 (+ 17us) server_negotiation.cc:299] Negotiation successful
0925 07:29:45.587077 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":70,"threads_started":1}
I20260925 07:29:45.587970 28339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:45.584440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48488 (local address 127.5.158.124:37331)
0925 07:29:45.584476 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:45.584478 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:45.584576 (+ 98us) server_negotiation.cc:407] Connection header received
0925 07:29:45.585161 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:45.585162 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:45.585173 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:45.585226 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 07:29:45.585402 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:45.586113 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:45.587366 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:45.587593 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:45.587636 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:45.587776 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 07:29:45.587803 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 07:29:45.587855 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:45.587886 (+ 31us) server_negotiation.cc:299] Negotiation successful
0925 07:29:45.587902 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260925 07:29:45.594477 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 27982
I20260925 07:29:45.599397 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 28057
I20260925 07:29:45.603998 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 28132
2026-09-25T07:29:45Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2665 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 07:29:45 dist-test-slave-1drq krb5kdc[28350](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 07:29:45 dist-test-slave-1drq krb5kdc[28350](info): set up 2 sockets
Sep 25 07:29:45 dist-test-slave-1drq krb5kdc[28350](info): commencing operation
krb5kdc: starting...
W20260925 07:29:46.568912 5753 mini_kdc.cc:121] Time spent starting KDC: real 0.946s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/krb5kdc/test-user.keytab.
Sep 25 07:29:46 dist-test-slave-1drq krb5kdc[28350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T07:29:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T07:29:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.158.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
Entry for principal kudu/127.5.158.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab.
I20260925 07:29:46.678094 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:35211
--webserver_interface=127.5.158.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:32813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.158.126:35211,127.5.158.125:33707,127.5.158.124:40101
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 07:29:46.750753 28363 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 07:29:46.750959 28363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:29:46.750986 28363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:29:46.751022 28363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:29:46.752070 28363 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:29:46.752140 28363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:29:46.752168 28363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:29:46.752195 28363 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:29:46.752216 28363 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:29:46.753603 28363 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:32813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.158.126:35211,127.5.158.125:33707,127.5.158.124:40101
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.126.keytab
--principal=kudu/127.5.158.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.126:35211
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.158.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28363
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:29:46.753831 28363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:29:46.754020 28363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:29:46.758100 28368 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:46.758132 28369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:46.758194 28363 server_base.cc:1034] running on GCE node
W20260925 07:29:46.758132 28371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:46.758576 28363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:29:46.758828 28363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:29:46.759999 28363 hybrid_clock.cc:648] HybridClock initialized: now 1790321386759974 us; error 41 us; skew 500 ppm
Sep 25 07:29:46 dist-test-slave-1drq krb5kdc[28350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321386, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:29:46.762455 28363 init.cc:377] Logged in from keytab as kudu/127.5.158.126@KRBTEST.COM (short username kudu)
I20260925 07:29:46.762844 28363 webserver.cc:460] Webserver started at http://127.5.158.126:39123/ using document root <none> and password file <none>
I20260925 07:29:46.763082 28363 fs_manager.cc:359] Metadata directory not provided
I20260925 07:29:46.763146 28363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:29:46.763283 28363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:29:46.764147 28363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/minicluster-data/master-0/data/instance:
uuid: "80189d3533634041980b5a67abfa7745"
format_stamp: "Formatted at 2026-09-25 07:29:46 on dist-test-slave-1drq"
server_key: "3d6225fbee46649293fb054049576880"
server_key_iv: "90401230579cf6b916c6750979cf43f5"
server_key_version: "encryptionkey@0"
I20260925 07:29:46.764468 28363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/minicluster-data/master-0/wal/instance:
uuid: "80189d3533634041980b5a67abfa7745"
format_stamp: "Formatted at 2026-09-25 07:29:46 on dist-test-slave-1drq"
server_key: "3d6225fbee46649293fb054049576880"
server_key_iv: "90401230579cf6b916c6750979cf43f5"
server_key_version: "encryptionkey@0"
I20260925 07:29:46.765699 28363 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260925 07:29:46.766325 28378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:46.766541 28363 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:29:46.766659 28363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/minicluster-data/master-0/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/minicluster-data/master-0/wal
uuid: "80189d3533634041980b5a67abfa7745"
format_stamp: "Formatted at 2026-09-25 07:29:46 on dist-test-slave-1drq"
server_key: "3d6225fbee46649293fb054049576880"
server_key_iv: "90401230579cf6b916c6750979cf43f5"
server_key_version: "encryptionkey@0"
I20260925 07:29:46.766737 28363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:29:46.775408 28363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:29:46.777467 28363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:29:46.777626 28363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:29:46.780938 28363 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.126:35211
I20260925 07:29:46.780977 28430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.126:35211 every 8 connection(s)
I20260925 07:29:46.781322 28363 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/minicluster-data/master-0/data/info.pb
I20260925 07:29:46.781736 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 28363
I20260925 07:29:46.781870 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/minicluster-data/master-0/wal/instance
I20260925 07:29:46.782058 5753 external_mini_cluster.cc:1428] Setting key 17480fd1c46c4eb8b9d12f6a637d42aa
I20260925 07:29:46.782325 28431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:29:46.783607 28431 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.158.126" port: 35211 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.5.158.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.125@KRBTEST.COM" created.
Sep 25 07:29:46 dist-test-slave-1drq krb5kdc[28350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321386, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.126@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:29:46.796188 28434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:46.784195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48218 (local address 127.5.158.126:35211)
0925 07:29:46.784341 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:46.784349 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:46.784432 (+ 83us) server_negotiation.cc:407] Connection header received
0925 07:29:46.784936 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:46.784939 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:46.784961 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:46.785060 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:46.785670 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:46.787012 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:46.787869 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:46.788339 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:46.788456 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:46.793210 (+ 4754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:46.793217 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:46.793217 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:29:46.793230 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:46.794699 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:46.795073 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:46.795075 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:46.795075 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:46.795161 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:46.795399 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:46.795400 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:46.795400 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:46.795579 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:46.795668 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:46.795851 (+ 183us) server_negotiation.cc:299] Negotiation successful
0925 07:29:46.795979 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":65,"threads_started":1}
I20260925 07:29:46.796188 28435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:46.784168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:35211 (local address 127.0.0.1:48218)
0925 07:29:46.784335 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:46.784372 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:46.784793 (+ 421us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:46.785057 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:46.785059 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:46.785410 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:29:46.785545 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:46.785549 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:46.787113 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:46.787114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:46.787778 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:46.787780 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:46.788436 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:46.788437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:46.788485 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:46.788889 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:29:46.788896 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:29:46.793076 (+ 4180us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:29:46.794774 (+ 1698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:29:46.794779 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:29:46.794780 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:29:46.794955 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:29:46.795230 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:29:46.795230 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:29:46.795231 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:29:46.795329 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:29:46.795651 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:29:46.795656 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:29:46.795710 (+ 54us) client_negotiation.cc:769] Sending connection context
0925 07:29:46.795819 (+ 109us) client_negotiation.cc:240] Negotiation successful
0925 07:29:46.796024 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":90,"thread_start_us":77,"threads_started":1}
I20260925 07:29:46.796926 28431 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.158.125" port: 33707 } has no permanent_uuid. Determining permanent_uuid...
I20260925 07:29:46.797273 28435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:46.797161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.125:33707 (local address 127.0.0.1:45448)
0925 07:29:46.797211 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:46.797250 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.125:33707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260925 07:29:46.797465 28381 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.158.125:33707: connect: Connection refused (error 111)
I20260925 07:29:46.797855 28435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:46.797774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.125:33707 (local address 127.0.0.1:45462)
0925 07:29:46.797821 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:46.797841 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.125:33707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260925 07:29:46.798003 28431 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.158.125:33707: Network error: Client connection negotiation failed: client connection to 127.5.158.125:33707: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.125.keytab.
Entry for principal kudu/127.5.158.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.125.keytab.
I20260925 07:29:46.806066 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.125:33707
--webserver_interface=127.5.158.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.125.keytab
--principal=kudu/127.5.158.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:32813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.158.126:35211,127.5.158.125:33707,127.5.158.124:40101
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 07:29:46.847177 28431 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.158.125" port: 33707 } attempt: 1
I20260925 07:29:46.847563 28435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:46.847436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.125:33707 (local address 127.0.0.1:45470)
0925 07:29:46.847495 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:46.847532 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.125:33707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260925 07:29:46.847918 28435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:46.847813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.125:33707 (local address 127.0.0.1:45472)
0925 07:29:46.847872 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:46.847900 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.125:33707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260925 07:29:46.848053 28431 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.158.125:33707: Network error: Client connection negotiation failed: client connection to 127.5.158.125:33707: connect: Connection refused (error 111)
W20260925 07:29:46.879079 28438 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 07:29:46.879309 28438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:29:46.879326 28438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:29:46.879345 28438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:29:46.880419 28438 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:29:46.880476 28438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:29:46.880522 28438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:29:46.880533 28438 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:29:46.880545 28438 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:29:46.882004 28438 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:32813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.158.126:35211,127.5.158.125:33707,127.5.158.124:40101
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.125.keytab
--principal=kudu/127.5.158.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.125:33707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.158.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28438
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:29:46.882229 28438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:29:46.882453 28438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 07:29:46.887039 28438 server_base.cc:1034] running on GCE node
W20260925 07:29:46.887024 28443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:46.887024 28446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:46.887030 28444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:46.887436 28438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:29:46.887662 28438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:29:46.888844 28438 hybrid_clock.cc:648] HybridClock initialized: now 1790321386888835 us; error 25 us; skew 500 ppm
Sep 25 07:29:46 dist-test-slave-1drq krb5kdc[28350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321386, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:29:46.891031 28438 init.cc:377] Logged in from keytab as kudu/127.5.158.125@KRBTEST.COM (short username kudu)
I20260925 07:29:46.891368 28438 webserver.cc:460] Webserver started at http://127.5.158.125:35389/ using document root <none> and password file <none>
I20260925 07:29:46.891523 28438 fs_manager.cc:359] Metadata directory not provided
I20260925 07:29:46.891559 28438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:29:46.891638 28438 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:29:46.892542 28438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/minicluster-data/master-1/data/instance:
uuid: "60eb036ef4b842de878eadc22233a7e9"
format_stamp: "Formatted at 2026-09-25 07:29:46 on dist-test-slave-1drq"
server_key: "4152ba24179b3c67f629acefb60e8047"
server_key_iv: "48e5e90735713943757e22a31193581e"
server_key_version: "encryptionkey@0"
I20260925 07:29:46.892933 28438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/minicluster-data/master-1/wal/instance:
uuid: "60eb036ef4b842de878eadc22233a7e9"
format_stamp: "Formatted at 2026-09-25 07:29:46 on dist-test-slave-1drq"
server_key: "4152ba24179b3c67f629acefb60e8047"
server_key_iv: "48e5e90735713943757e22a31193581e"
server_key_version: "encryptionkey@0"
I20260925 07:29:46.894177 28438 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 07:29:46.894868 28453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:46.895041 28438 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:29:46.895100 28438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/minicluster-data/master-1/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/minicluster-data/master-1/wal
uuid: "60eb036ef4b842de878eadc22233a7e9"
format_stamp: "Formatted at 2026-09-25 07:29:46 on dist-test-slave-1drq"
server_key: "4152ba24179b3c67f629acefb60e8047"
server_key_iv: "48e5e90735713943757e22a31193581e"
server_key_version: "encryptionkey@0"
I20260925 07:29:46.895180 28438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:29:46.907330 28438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:29:46.908836 28438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:29:46.909000 28438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:29:46.912324 28438 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.125:33707
I20260925 07:29:46.912416 28505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.125:33707 every 8 connection(s)
I20260925 07:29:46.912693 28438 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/minicluster-data/master-1/data/info.pb
I20260925 07:29:46.913524 28506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:29:46.914788 28506 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.158.126" port: 35211 } has no permanent_uuid. Determining permanent_uuid...
I20260925 07:29:46.916206 28431 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.158.125" port: 33707 } attempt: 2
I20260925 07:29:46.919672 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 28438
I20260925 07:29:46.919797 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/minicluster-data/master-1/wal/instance
I20260925 07:29:46.920010 5753 external_mini_cluster.cc:1428] Setting key 6b78900e3db1164ddc0386c59c24aa6d
Sep 25 07:29:46 dist-test-slave-1drq krb5kdc[28350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321386, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.126@KRBTEST.COM for kudu/127.5.158.125@KRBTEST.COM
Sep 25 07:29:46 dist-test-slave-1drq krb5kdc[28350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321386, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.125@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:29:46.924782 28434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:46.915395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48226 (local address 127.5.158.126:35211)
0925 07:29:46.915471 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:46.915473 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:46.915615 (+ 142us) server_negotiation.cc:407] Connection header received
0925 07:29:46.916034 (+ 419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:46.916035 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:46.916048 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:46.916092 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:46.916777 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:46.917359 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:46.918514 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:46.918773 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:46.918836 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:46.921817 (+ 2981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:46.921819 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:46.921819 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:29:46.921827 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:46.923395 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:46.923801 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:46.923802 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:46.923802 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:46.923849 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:46.924098 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:46.924098 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:46.924099 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:46.924203 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:46.924282 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:46.924542 (+ 260us) server_negotiation.cc:299] Negotiation successful
0925 07:29:46.924645 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 07:29:46.924929 28509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:46.916587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45484 (local address 127.5.158.125:33707)
0925 07:29:46.916726 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:46.916729 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:46.916742 (+ 13us) server_negotiation.cc:407] Connection header received
0925 07:29:46.916759 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:46.916760 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:46.916773 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:46.916825 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:46.917135 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:46.917856 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:46.918683 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:46.919028 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:46.919084 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:46.921471 (+ 2387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:46.921478 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:46.921479 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:29:46.921485 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:46.923389 (+ 1904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:46.923761 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:46.923764 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:46.923764 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:46.923864 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:46.924079 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:46.924080 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:46.924080 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:46.924357 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:46.924442 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:46.924765 (+ 323us) server_negotiation.cc:299] Negotiation successful
0925 07:29:46.924818 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":77,"threads_started":1}
I20260925 07:29:46.924939 28435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:46.916477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.125:33707 (local address 127.0.0.1:45484)
0925 07:29:46.916541 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:46.916552 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:46.916592 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:46.916849 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:46.916850 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:46.916995 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:29:46.917040 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:46.917041 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:46.917952 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:46.917953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:46.918611 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:46.918613 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:46.919115 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:46.919116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:46.919201 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:46.919472 (+ 271us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:29:46.919476 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:29:46.921372 (+ 1896us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:29:46.923466 (+ 2094us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:29:46.923467 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:29:46.923468 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:29:46.923667 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:29:46.923955 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:29:46.923955 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:29:46.923956 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:29:46.924005 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:29:46.924442 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:29:46.924443 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:29:46.924498 (+ 55us) client_negotiation.cc:769] Sending connection context
0925 07:29:46.924557 (+ 59us) client_negotiation.cc:240] Negotiation successful
0925 07:29:46.924693 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":52}
I20260925 07:29:46.925338 28508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:46.915324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:35211 (local address 127.0.0.1:48226)
0925 07:29:46.915514 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:46.915543 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:46.915937 (+ 394us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:46.916150 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:46.916153 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:46.916396 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:29:46.916681 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:46.916687 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:46.917441 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:46.917442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:46.918433 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:46.918436 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:46.918867 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:46.918868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:46.918955 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:46.919394 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:29:46.919429 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:29:46.921705 (+ 2276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:29:46.923500 (+ 1795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:29:46.923506 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:29:46.923508 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:29:46.923741 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:29:46.923925 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:29:46.923925 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:29:46.923926 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:29:46.924018 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:29:46.924275 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:29:46.924281 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:29:46.924349 (+ 68us) client_negotiation.cc:769] Sending connection context
0925 07:29:46.924511 (+ 162us) client_negotiation.cc:240] Negotiation successful
0925 07:29:46.924656 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":99,"thread_start_us":69,"threads_started":1}
I20260925 07:29:46.925439 28431 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.158.124" port: 40101 } has no permanent_uuid. Determining permanent_uuid...
I20260925 07:29:46.925774 28435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:46.925606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.124:40101 (local address 127.0.0.1:41346)
0925 07:29:46.925728 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:46.925753 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.124:40101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260925 07:29:46.925873 28506 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.158.125" port: 33707 } has no permanent_uuid. Determining permanent_uuid...
I20260925 07:29:46.926066 28435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:46.925981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.124:40101 (local address 127.0.0.1:41354)
0925 07:29:46.926010 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:46.926039 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.124:40101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260925 07:29:46.926141 28431 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.158.124:40101: Network error: Client connection negotiation failed: client connection to 127.5.158.124:40101: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.5.158.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.158.124@KRBTEST.COM" created.
Sep 25 07:29:46 dist-test-slave-1drq krb5kdc[28350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321386, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.125@KRBTEST.COM for kudu/127.5.158.125@KRBTEST.COM
I20260925 07:29:46.934144 28508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:46.926038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.125:33707 (local address 127.0.0.1:45498)
0925 07:29:46.926094 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:46.926101 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:46.926133 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:46.926240 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:46.926240 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:46.926375 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:29:46.926395 (+ 20us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:46.926397 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:46.926996 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:46.926997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:46.927437 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:46.927438 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:46.927827 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:46.927828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:46.927880 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:46.928318 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:29:46.928323 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:29:46.931420 (+ 3097us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:29:46.933313 (+ 1893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:29:46.933314 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:29:46.933314 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:29:46.933450 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:29:46.933634 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:29:46.933634 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:29:46.933634 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:29:46.933675 (+ 41us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:29:46.933942 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:29:46.933943 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:29:46.933985 (+ 42us) client_negotiation.cc:769] Sending connection context
0925 07:29:46.934043 (+ 58us) client_negotiation.cc:240] Negotiation successful
0925 07:29:46.934098 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260925 07:29:46.934285 28509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:46.926067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45498 (local address 127.5.158.125:33707)
0925 07:29:46.926092 (+ 25us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:46.926093 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:46.926173 (+ 80us) server_negotiation.cc:407] Connection header received
0925 07:29:46.926186 (+ 13us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:46.926187 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:46.926194 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:46.926226 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:46.926508 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:46.926922 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:46.927538 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:46.927744 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:46.927827 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:46.931518 (+ 3691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:46.931520 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:46.931520 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:29:46.931526 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:46.933243 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:46.933520 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:46.933521 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:46.933521 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:46.933573 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:46.933739 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:46.933740 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:46.933740 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:46.933856 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:46.933920 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:46.934095 (+ 175us) server_negotiation.cc:299] Negotiation successful
0925 07:29:46.934202 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10}
I20260925 07:29:46.934576 28506 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.158.124" port: 40101 } has no permanent_uuid. Determining permanent_uuid...
I20260925 07:29:46.934849 28508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:46.934752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.124:40101 (local address 127.0.0.1:41356)
0925 07:29:46.934797 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:46.934831 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.124:40101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260925 07:29:46.935002 28455 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.158.124:40101: connect: Connection refused (error 111)
I20260925 07:29:46.935350 28508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:46.935244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.124:40101 (local address 127.0.0.1:41370)
0925 07:29:46.935308 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:46.935335 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.124:40101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260925 07:29:46.935452 28506 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.158.124:40101: Network error: Client connection negotiation failed: client connection to 127.5.158.124:40101: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.158.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.124.keytab.
Entry for principal kudu/127.5.158.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.124.keytab.
I20260925 07:29:46.946146 5753 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskROew7s/build/release/bin/kudu
/tmp/dist-test-taskROew7s/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.124:40101
--webserver_interface=127.5.158.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.124.keytab
--principal=kudu/127.5.158.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.158.84:32813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.158.126:35211,127.5.158.125:33707,127.5.158.124:40101
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 07:29:46.969235 28431 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.158.124" port: 40101 } attempt: 1
I20260925 07:29:46.969548 28435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:46.969447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.124:40101 (local address 127.0.0.1:41378)
0925 07:29:46.969496 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:46.969526 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.124:40101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260925 07:29:46.969870 28435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:46.969784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.124:40101 (local address 127.0.0.1:41386)
0925 07:29:46.969831 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:46.969853 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.124:40101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260925 07:29:46.970001 28431 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.158.124:40101: Network error: Client connection negotiation failed: client connection to 127.5.158.124:40101: connect: Connection refused (error 111)
I20260925 07:29:46.984771 28506 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.158.124" port: 40101 } attempt: 1
I20260925 07:29:46.985126 28508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:46.984984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.124:40101 (local address 127.0.0.1:41390)
0925 07:29:46.985036 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:46.985065 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.124:40101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260925 07:29:46.985400 28508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:46.985313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.124:40101 (local address 127.0.0.1:41392)
0925 07:29:46.985356 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:46.985377 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.124:40101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260925 07:29:46.985476 28506 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.158.124:40101: Network error: Client connection negotiation failed: client connection to 127.5.158.124:40101: connect: Connection refused (error 111)
I20260925 07:29:47.017162 28431 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.158.124" port: 40101 } attempt: 2
I20260925 07:29:47.017506 28435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:47.017401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.124:40101 (local address 127.0.0.1:41400)
0925 07:29:47.017456 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:47.017486 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.124:40101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260925 07:29:47.017844 28435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:47.017770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.124:40101 (local address 127.0.0.1:41408)
0925 07:29:47.017812 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:47.017831 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.124:40101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260925 07:29:47.017930 28431 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.158.124:40101: Network error: Client connection negotiation failed: client connection to 127.5.158.124:40101: connect: Connection refused (error 111)
W20260925 07:29:47.018211 28513 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 07:29:47.018390 28513 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 07:29:47.018420 28513 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 07:29:47.018447 28513 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 07:29:47.019450 28513 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 07:29:47.019516 28513 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 07:29:47.019539 28513 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 07:29:47.019562 28513 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 07:29:47.019585 28513 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 07:29:47.020968 28513 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.158.84:32813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.158.126:35211,127.5.158.125:33707,127.5.158.124:40101
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/krb5kdc/kudu_127.5.158.124.keytab
--principal=kudu/127.5.158.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.158.124:40101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.158.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28513
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 06:22:12 UTC on ccycloud.quasar-vmwxne.root.comops.site
build id 4671942
I20260925 07:29:47.021245 28513 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 07:29:47.021463 28513 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 07:29:47.026003 28521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 07:29:47.026046 28518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:47.026104 28513 server_base.cc:1034] running on GCE node
W20260925 07:29:47.026013 28519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 07:29:47.026345 28513 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 07:29:47.026618 28513 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 07:29:47.027791 28513 hybrid_clock.cc:648] HybridClock initialized: now 1790321387027778 us; error 25 us; skew 500 ppm
Sep 25 07:29:47 dist-test-slave-1drq krb5kdc[28350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321387, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 07:29:47.030043 28513 init.cc:377] Logged in from keytab as kudu/127.5.158.124@KRBTEST.COM (short username kudu)
I20260925 07:29:47.030442 28513 webserver.cc:460] Webserver started at http://127.5.158.124:33927/ using document root <none> and password file <none>
I20260925 07:29:47.030607 28513 fs_manager.cc:359] Metadata directory not provided
I20260925 07:29:47.030642 28513 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 07:29:47.030716 28513 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 07:29:47.031538 28513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/minicluster-data/master-2/data/instance:
uuid: "816b53b67748499a96354d14e4696b32"
format_stamp: "Formatted at 2026-09-25 07:29:47 on dist-test-slave-1drq"
server_key: "33f24abc406f99cfc5c253bae0d5974c"
server_key_iv: "399a549f8dd3c296767614e3553b3af0"
server_key_version: "encryptionkey@0"
I20260925 07:29:47.031823 28513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/minicluster-data/master-2/wal/instance:
uuid: "816b53b67748499a96354d14e4696b32"
format_stamp: "Formatted at 2026-09-25 07:29:47 on dist-test-slave-1drq"
server_key: "33f24abc406f99cfc5c253bae0d5974c"
server_key_iv: "399a549f8dd3c296767614e3553b3af0"
server_key_version: "encryptionkey@0"
I20260925 07:29:47.033016 28513 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 07:29:47.033739 28528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 07:29:47.033900 28513 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 07:29:47.033978 28513 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/minicluster-data/master-2/data,/tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/minicluster-data/master-2/wal
uuid: "816b53b67748499a96354d14e4696b32"
format_stamp: "Formatted at 2026-09-25 07:29:47 on dist-test-slave-1drq"
server_key: "33f24abc406f99cfc5c253bae0d5974c"
server_key_iv: "399a549f8dd3c296767614e3553b3af0"
server_key_version: "encryptionkey@0"
I20260925 07:29:47.034024 28513 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 07:29:47.053602 28506 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.158.124" port: 40101 } attempt: 2
I20260925 07:29:47.053962 28508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:47.053841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.124:40101 (local address 127.0.0.1:41420)
0925 07:29:47.053912 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:47.053941 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.124:40101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260925 07:29:47.054302 28508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:47.054218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.124:40101 (local address 127.0.0.1:41426)
0925 07:29:47.054264 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:47.054286 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.158.124:40101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260925 07:29:47.054409 28506 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.158.124:40101: Network error: Client connection negotiation failed: client connection to 127.5.158.124:40101: connect: Connection refused (error 111)
I20260925 07:29:47.054708 28513 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 07:29:47.056685 28513 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 07:29:47.056823 28513 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 07:29:47.060288 28513 rpc_server.cc:307] RPC server started. Bound to: 127.5.158.124:40101
I20260925 07:29:47.060315 28580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.158.124:40101 every 8 connection(s)
I20260925 07:29:47.060664 28513 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/minicluster-data/master-2/data/info.pb
I20260925 07:29:47.061581 28581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 07:29:47.062961 28581 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.158.126" port: 35211 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 07:29:47 dist-test-slave-1drq krb5kdc[28350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321387, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.124@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:29:47.070113 5753 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskROew7s/build/release/bin/kudu as pid 28513
I20260925 07:29:47.070218 5753 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskROew7s/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790321110323796-5753-0/minicluster-data/master-2/wal/instance
I20260925 07:29:47.070408 5753 external_mini_cluster.cc:1428] Setting key 19d860966a45b3e5efe87990caffbd66
I20260925 07:29:47.070991 28434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:47.063551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48232 (local address 127.5.158.126:35211)
0925 07:29:47.063613 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:47.063615 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:47.063798 (+ 183us) server_negotiation.cc:407] Connection header received
0925 07:29:47.064155 (+ 357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:47.064156 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:47.064164 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:47.064209 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:47.064675 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:47.065081 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:47.066026 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:47.066218 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:47.066266 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:47.067959 (+ 1693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:47.067961 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:47.067961 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:29:47.067966 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:47.069629 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:47.069974 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:47.069975 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:47.069975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:47.070016 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:47.070313 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:47.070314 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:47.070314 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:47.070421 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:47.070495 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:47.070810 (+ 315us) server_negotiation.cc:299] Negotiation successful
0925 07:29:47.070888 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260925 07:29:47.071162 28583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:47.063538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.126:35211 (local address 127.0.0.1:48232)
0925 07:29:47.063692 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:47.063720 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:47.064086 (+ 366us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:47.064261 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:47.064263 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:47.064454 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:29:47.064616 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:47.064619 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:47.065209 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:47.065210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:47.065954 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:47.065956 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:47.066300 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:47.066301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:47.066399 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:47.066753 (+ 354us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:29:47.066760 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:29:47.067884 (+ 1124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:29:47.069688 (+ 1804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:29:47.069693 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:29:47.069694 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:29:47.069896 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:29:47.070111 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:29:47.070112 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:29:47.070113 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:29:47.070255 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:29:47.070502 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:29:47.070508 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:29:47.070641 (+ 133us) client_negotiation.cc:769] Sending connection context
0925 07:29:47.070776 (+ 135us) client_negotiation.cc:240] Negotiation successful
0925 07:29:47.070931 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":83,"threads_started":1}
I20260925 07:29:47.071640 28581 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.158.125" port: 33707 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 07:29:47 dist-test-slave-1drq krb5kdc[28350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321387, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.124@KRBTEST.COM for kudu/127.5.158.125@KRBTEST.COM
Sep 25 07:29:47 dist-test-slave-1drq krb5kdc[28350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.158.124@KRBTEST.COM
I20260925 07:29:47.078653 28583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:47.071875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.125:33707 (local address 127.0.0.1:45508)
0925 07:29:47.071936 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:47.071945 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:47.071986 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:47.072144 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:47.072145 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:47.072244 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:29:47.072284 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:47.072285 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:47.073119 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:47.073120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:47.073796 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:47.073798 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:47.074214 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:47.074214 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:47.074261 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:47.074598 (+ 337us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:29:47.074602 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:29:47.075705 (+ 1103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:29:47.077592 (+ 1887us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:29:47.077593 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:29:47.077594 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:29:47.077904 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:29:47.078094 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:29:47.078095 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:29:47.078095 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:29:47.078158 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:29:47.078389 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:29:47.078390 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:29:47.078430 (+ 40us) client_negotiation.cc:769] Sending connection context
0925 07:29:47.078513 (+ 83us) client_negotiation.cc:240] Negotiation successful
0925 07:29:47.078593 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":1}
I20260925 07:29:47.078703 28509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:47.071970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45508 (local address 127.5.158.125:33707)
0925 07:29:47.072040 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:47.072042 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:47.072050 (+ 8us) server_negotiation.cc:407] Connection header received
0925 07:29:47.072073 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:47.072074 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:47.072083 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:47.072143 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:47.072353 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:47.073037 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:47.073865 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:47.074144 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:47.074208 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:47.075788 (+ 1580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:47.075790 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:47.075790 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:29:47.075795 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:47.077485 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:47.077986 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:47.077987 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:47.077987 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:47.078030 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:47.078233 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:47.078234 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:47.078234 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:47.078328 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:47.078369 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:47.078551 (+ 182us) server_negotiation.cc:299] Negotiation successful
0925 07:29:47.078616 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 07:29:47.079037 28581 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.158.124" port: 40101 } has no permanent_uuid. Determining permanent_uuid...
I20260925 07:29:47.079891 28585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:47.070756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41432 (local address 127.5.158.124:40101)
0925 07:29:47.070881 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:47.070885 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:47.071012 (+ 127us) server_negotiation.cc:407] Connection header received
0925 07:29:47.071039 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:47.071041 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:47.071052 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:47.071123 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:47.071801 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:47.072842 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:47.073866 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:47.074273 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:47.074325 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:47.076420 (+ 2095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:47.076427 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:47.076427 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:29:47.076451 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:47.078434 (+ 1983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:47.078988 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:47.078991 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:47.078991 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:47.079050 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:47.079264 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:47.079265 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:47.079265 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:47.079436 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:47.079590 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:47.079686 (+ 96us) server_negotiation.cc:299] Negotiation successful
0925 07:29:47.079797 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":77,"threads_started":1}
Sep 25 07:29:47 dist-test-slave-1drq krb5kdc[28350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321387, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.124@KRBTEST.COM for kudu/127.5.158.124@KRBTEST.COM
Sep 25 07:29:47 dist-test-slave-1drq krb5kdc[28350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.158.126@KRBTEST.COM
I20260925 07:29:47.086516 28583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:47.079215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.124:40101 (local address 127.0.0.1:41436)
0925 07:29:47.079324 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:47.079331 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:47.079354 (+ 23us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:47.079529 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:47.079530 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:47.079674 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:29:47.079712 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:47.079714 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:47.080386 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:47.080386 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:47.081234 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:47.081237 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:47.081609 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:47.081610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:47.081656 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:47.082039 (+ 383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:29:47.082043 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:29:47.083170 (+ 1127us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:29:47.085419 (+ 2249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:29:47.085420 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:29:47.085421 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:29:47.085636 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:29:47.085881 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:29:47.085882 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:29:47.085882 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:29:47.085942 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:29:47.086181 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:29:47.086181 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:29:47.086230 (+ 49us) client_negotiation.cc:769] Sending connection context
0925 07:29:47.086324 (+ 94us) client_negotiation.cc:240] Negotiation successful
0925 07:29:47.086411 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99}
I20260925 07:29:47.086609 28586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:47.079241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41436 (local address 127.5.158.124:40101)
0925 07:29:47.079367 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:47.079369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:47.079377 (+ 8us) server_negotiation.cc:407] Connection header received
0925 07:29:47.079395 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:47.079395 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:47.079404 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:47.079500 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:47.079788 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:47.080326 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:47.081321 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:47.081511 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:47.081607 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:47.083263 (+ 1656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:47.083264 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:47.083265 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:29:47.083271 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:47.085330 (+ 2059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:47.085734 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:47.085735 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:47.085735 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:47.085794 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:47.086012 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:47.086013 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:47.086014 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:47.086112 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:47.086188 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:47.086346 (+ 158us) server_negotiation.cc:299] Negotiation successful
0925 07:29:47.086456 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":53,"server-negotiator.queue_time_us":106,"thread_start_us":57,"threads_started":1}
I20260925 07:29:47.088074 28581 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 816b53b67748499a96354d14e4696b32: Bootstrap starting.
I20260925 07:29:47.088182 28434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:47.080822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48236 (local address 127.5.158.126:35211)
0925 07:29:47.080855 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:47.080857 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:47.080865 (+ 8us) server_negotiation.cc:407] Connection header received
0925 07:29:47.080885 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:47.080885 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:47.080894 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:47.080956 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:47.081508 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:47.082074 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:47.082753 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:47.083017 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:47.083075 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:47.085646 (+ 2571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:47.085646 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:47.085647 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:29:47.085652 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:47.087210 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:47.087504 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:47.087505 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:47.087506 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:47.087566 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:47.087711 (+ 145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:47.087712 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:47.087712 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:47.087827 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:47.087986 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:47.088027 (+ 41us) server_negotiation.cc:299] Negotiation successful
0925 07:29:47.088098 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 07:29:47.088750 28581 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 816b53b67748499a96354d14e4696b32: Neither blocks nor log segments found. Creating new log.
I20260925 07:29:47.089160 28581 log.cc:824] T 00000000000000000000000000000000 P 816b53b67748499a96354d14e4696b32: Log is configured to *not* fsync() on all Append() calls
I20260925 07:29:47.089915 28581 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 816b53b67748499a96354d14e4696b32: No bootstrap required, opened a new log
I20260925 07:29:47.091760 28581 raft_consensus.cc:348] T 00000000000000000000000000000000 P 816b53b67748499a96354d14e4696b32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80189d3533634041980b5a67abfa7745" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 35211 } } peers { permanent_uuid: "60eb036ef4b842de878eadc22233a7e9" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 33707 } } peers { permanent_uuid: "816b53b67748499a96354d14e4696b32" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 40101 } }
I20260925 07:29:47.091930 28581 raft_consensus.cc:729] T 00000000000000000000000000000000 P 816b53b67748499a96354d14e4696b32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 816b53b67748499a96354d14e4696b32, State: Initialized, Role: FOLLOWER
I20260925 07:29:47.092020 28581 consensus_queue.cc:260] T 00000000000000000000000000000000 P 816b53b67748499a96354d14e4696b32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80189d3533634041980b5a67abfa7745" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 35211 } } peers { permanent_uuid: "60eb036ef4b842de878eadc22233a7e9" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 33707 } } peers { permanent_uuid: "816b53b67748499a96354d14e4696b32" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 40101 } }
I20260925 07:29:47.092315 28588 sys_catalog.cc:455] T 00000000000000000000000000000000 P 816b53b67748499a96354d14e4696b32 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80189d3533634041980b5a67abfa7745" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 35211 } } peers { permanent_uuid: "60eb036ef4b842de878eadc22233a7e9" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 33707 } } peers { permanent_uuid: "816b53b67748499a96354d14e4696b32" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 40101 } } }
I20260925 07:29:47.092368 28581 sys_catalog.cc:564] T 00000000000000000000000000000000 P 816b53b67748499a96354d14e4696b32 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:29:47.092442 28588 sys_catalog.cc:458] T 00000000000000000000000000000000 P 816b53b67748499a96354d14e4696b32 [sys.catalog]: This master's current role is: FOLLOWER
Sep 25 07:29:47 dist-test-slave-1drq krb5kdc[28350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.158.125@KRBTEST.COM
W20260925 07:29:47.094091 28599 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 816b53b67748499a96354d14e4696b32: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 07:29:47.094177 28599 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 07:29:47.096486 28509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:47.089180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45520 (local address 127.5.158.125:33707)
0925 07:29:47.089219 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:47.089221 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:47.089242 (+ 21us) server_negotiation.cc:407] Connection header received
0925 07:29:47.089259 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:47.089260 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:47.089269 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:47.089314 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:47.089899 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:47.090308 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:47.091132 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:47.091294 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:47.091337 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:47.093883 (+ 2546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:47.093884 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:47.093885 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:29:47.093891 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:47.095377 (+ 1486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:47.095760 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:47.095761 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:47.095761 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:47.095814 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:47.096023 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:47.096024 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:47.096024 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:47.096132 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:47.096260 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:47.096347 (+ 87us) server_negotiation.cc:299] Negotiation successful
0925 07:29:47.096416 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 07:29:47.094712 28513 master_runner.cc:186] Error getting master registration for 127.5.158.126:35211: OK, instance_id { permanent_uuid: "80189d3533634041980b5a67abfa7745" instance_seqno: 1790321386780000 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 07:29:47.096777 28513 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 07:29:47.097709 5753 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 07:29:47.098464 28405 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "80189d3533634041980b5a67abfa7745"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48236
I20260925 07:29:47.125063 28431 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.158.124" port: 40101 } attempt: 3
Sep 25 07:29:47 dist-test-slave-1drq krb5kdc[28350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321386, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.126@KRBTEST.COM for kudu/127.5.158.124@KRBTEST.COM
I20260925 07:29:47.131980 28435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:47.125370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.124:40101 (local address 127.0.0.1:41452)
0925 07:29:47.125416 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:47.125425 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:47.125456 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:47.125625 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:47.125626 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:47.125813 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:29:47.125854 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:47.125856 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:47.126706 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:47.126706 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:47.127407 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:47.127409 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:47.127852 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:47.127853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:47.127897 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:47.128270 (+ 373us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:29:47.128274 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:29:47.129309 (+ 1035us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:29:47.130899 (+ 1590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:29:47.130900 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:29:47.130900 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:29:47.131121 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:29:47.131310 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:29:47.131310 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:29:47.131311 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:29:47.131387 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:29:47.131673 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:29:47.131674 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:29:47.131714 (+ 40us) client_negotiation.cc:769] Sending connection context
0925 07:29:47.131787 (+ 73us) client_negotiation.cc:240] Negotiation successful
0925 07:29:47.131889 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260925 07:29:47.131997 28586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:47.125496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41452 (local address 127.5.158.124:40101)
0925 07:29:47.125535 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:47.125537 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:47.125543 (+ 6us) server_negotiation.cc:407] Connection header received
0925 07:29:47.125561 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:47.125561 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:47.125570 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:47.125619 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:47.125935 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:47.126624 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:47.127487 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:47.127778 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:47.127841 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:47.129389 (+ 1548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:47.129390 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:47.129390 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:29:47.129396 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:47.130821 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:47.131191 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:47.131192 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:47.131193 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:47.131243 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:47.131449 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:47.131450 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:47.131450 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:47.131603 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:47.131670 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:47.131843 (+ 173us) server_negotiation.cc:299] Negotiation successful
0925 07:29:47.131916 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 07:29:47.133241 28431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80189d3533634041980b5a67abfa7745: Bootstrap starting.
I20260925 07:29:47.133787 28431 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 80189d3533634041980b5a67abfa7745: Neither blocks nor log segments found. Creating new log.
I20260925 07:29:47.134042 28431 log.cc:824] T 00000000000000000000000000000000 P 80189d3533634041980b5a67abfa7745: Log is configured to *not* fsync() on all Append() calls
I20260925 07:29:47.134773 28431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80189d3533634041980b5a67abfa7745: No bootstrap required, opened a new log
I20260925 07:29:47.135941 28431 raft_consensus.cc:348] T 00000000000000000000000000000000 P 80189d3533634041980b5a67abfa7745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80189d3533634041980b5a67abfa7745" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 35211 } } peers { permanent_uuid: "60eb036ef4b842de878eadc22233a7e9" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 33707 } } peers { permanent_uuid: "816b53b67748499a96354d14e4696b32" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 40101 } }
I20260925 07:29:47.136096 28431 raft_consensus.cc:729] T 00000000000000000000000000000000 P 80189d3533634041980b5a67abfa7745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80189d3533634041980b5a67abfa7745, State: Initialized, Role: FOLLOWER
I20260925 07:29:47.136178 28431 consensus_queue.cc:260] T 00000000000000000000000000000000 P 80189d3533634041980b5a67abfa7745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80189d3533634041980b5a67abfa7745" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 35211 } } peers { permanent_uuid: "60eb036ef4b842de878eadc22233a7e9" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 33707 } } peers { permanent_uuid: "816b53b67748499a96354d14e4696b32" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 40101 } }
I20260925 07:29:47.136411 28602 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80189d3533634041980b5a67abfa7745 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80189d3533634041980b5a67abfa7745" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 35211 } } peers { permanent_uuid: "60eb036ef4b842de878eadc22233a7e9" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 33707 } } peers { permanent_uuid: "816b53b67748499a96354d14e4696b32" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 40101 } } }
I20260925 07:29:47.136449 28431 sys_catalog.cc:564] T 00000000000000000000000000000000 P 80189d3533634041980b5a67abfa7745 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:29:47.136485 28602 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80189d3533634041980b5a67abfa7745 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 07:29:47.136570 28405 raft_consensus.cc:482] T 00000000000000000000000000000000 P 80189d3533634041980b5a67abfa7745 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260925 07:29:47.136662 28405 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 80189d3533634041980b5a67abfa7745 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:29:47.137511 28405 raft_consensus.cc:504] T 00000000000000000000000000000000 P 80189d3533634041980b5a67abfa7745 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80189d3533634041980b5a67abfa7745" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 35211 } } peers { permanent_uuid: "60eb036ef4b842de878eadc22233a7e9" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 33707 } } peers { permanent_uuid: "816b53b67748499a96354d14e4696b32" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 40101 } }
I20260925 07:29:47.138214 28555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "80189d3533634041980b5a67abfa7745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "816b53b67748499a96354d14e4696b32"
I20260925 07:29:47.138311 28555 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 816b53b67748499a96354d14e4696b32 [term 0 FOLLOWER]: Advancing to term 1
W20260925 07:29:47.138443 28613 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 80189d3533634041980b5a67abfa7745: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 07:29:47.138489 28613 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 07:29:47.139026 28555 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 816b53b67748499a96354d14e4696b32 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80189d3533634041980b5a67abfa7745 in term 1.
I20260925 07:29:47.139261 28380 leader_election.cc:302] T 00000000000000000000000000000000 P 80189d3533634041980b5a67abfa7745 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80189d3533634041980b5a67abfa7745, 816b53b67748499a96354d14e4696b32; no voters:
I20260925 07:29:47.139355 28602 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 80189d3533634041980b5a67abfa7745 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 07:29:47.139467 28602 raft_consensus.cc:686] T 00000000000000000000000000000000 P 80189d3533634041980b5a67abfa7745 [term 1 LEADER]: Becoming Leader. State: Replica: 80189d3533634041980b5a67abfa7745, State: Running, Role: LEADER
I20260925 07:29:47.139531 28602 consensus_queue.cc:237] T 00000000000000000000000000000000 P 80189d3533634041980b5a67abfa7745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80189d3533634041980b5a67abfa7745" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 35211 } } peers { permanent_uuid: "60eb036ef4b842de878eadc22233a7e9" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 33707 } } peers { permanent_uuid: "816b53b67748499a96354d14e4696b32" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 40101 } }
I20260925 07:29:47.139864 28602 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80189d3533634041980b5a67abfa7745 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 80189d3533634041980b5a67abfa7745. Latest consensus state: current_term: 1 leader_uuid: "80189d3533634041980b5a67abfa7745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80189d3533634041980b5a67abfa7745" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 35211 } } peers { permanent_uuid: "60eb036ef4b842de878eadc22233a7e9" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 33707 } } peers { permanent_uuid: "816b53b67748499a96354d14e4696b32" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 40101 } } }
I20260925 07:29:47.139940 28602 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80189d3533634041980b5a67abfa7745 [sys.catalog]: This master's current role is: LEADER
I20260925 07:29:47.140192 28405 leader_election.cc:288] T 00000000000000000000000000000000 P 80189d3533634041980b5a67abfa7745 [CANDIDATE]: Term 1 election: Requested vote from peers 60eb036ef4b842de878eadc22233a7e9 (127.5.158.125:33707), 816b53b67748499a96354d14e4696b32 (127.5.158.124:40101)
I20260925 07:29:47.138813 28363 master_runner.cc:186] Error getting master registration for 127.5.158.125:33707: OK, instance_id { permanent_uuid: "60eb036ef4b842de878eadc22233a7e9" instance_seqno: 1790321386911402 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 07:29:47.140843 28363 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 07:29:47.141348 28617 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 07:29:47.141530 28617 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 07:29:47.143337 28555 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 816b53b67748499a96354d14e4696b32 [term 1 FOLLOWER]: Refusing update from remote peer 80189d3533634041980b5a67abfa7745: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:29:47.143695 28615 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 80189d3533634041980b5a67abfa7745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "816b53b67748499a96354d14e4696b32" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 40101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:29:47.144795 28588 sys_catalog.cc:455] T 00000000000000000000000000000000 P 816b53b67748499a96354d14e4696b32 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 80189d3533634041980b5a67abfa7745. Latest consensus state: current_term: 1 leader_uuid: "80189d3533634041980b5a67abfa7745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80189d3533634041980b5a67abfa7745" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 35211 } } peers { permanent_uuid: "60eb036ef4b842de878eadc22233a7e9" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 33707 } } peers { permanent_uuid: "816b53b67748499a96354d14e4696b32" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 40101 } } }
I20260925 07:29:47.144865 28588 sys_catalog.cc:458] T 00000000000000000000000000000000 P 816b53b67748499a96354d14e4696b32 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 07:29:47.145586 28506 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.158.124" port: 40101 } attempt: 3
Sep 25 07:29:47 dist-test-slave-1drq krb5kdc[28350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790321386, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.158.125@KRBTEST.COM for kudu/127.5.158.124@KRBTEST.COM
I20260925 07:29:47.154677 28508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:47.145781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.158.124:40101 (local address 127.0.0.1:41464)
0925 07:29:47.145843 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0925 07:29:47.145852 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 07:29:47.145883 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 07:29:47.146090 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 07:29:47.146090 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 07:29:47.146245 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0925 07:29:47.146282 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:47.146284 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:47.147154 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:47.147154 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:47.147961 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 07:29:47.147964 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 07:29:47.148508 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 07:29:47.148509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 07:29:47.148576 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:47.149208 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 07:29:47.149213 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 07:29:47.150523 (+ 1310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 07:29:47.153682 (+ 3159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:29:47.153683 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:29:47.153683 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:29:47.153885 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:29:47.154094 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 07:29:47.154095 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 07:29:47.154096 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 07:29:47.154156 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 07:29:47.154415 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 07:29:47.154415 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 07:29:47.154462 (+ 47us) client_negotiation.cc:769] Sending connection context
0925 07:29:47.154515 (+ 53us) client_negotiation.cc:240] Negotiation successful
0925 07:29:47.154593 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260925 07:29:47.155007 28585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:47.145940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41464 (local address 127.5.158.124:40101)
0925 07:29:47.145968 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:47.145969 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:47.145975 (+ 6us) server_negotiation.cc:407] Connection header received
0925 07:29:47.145990 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:47.145991 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:47.145998 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:47.146042 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:47.146370 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:47.147058 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:47.148067 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:47.148378 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:47.148437 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:47.150615 (+ 2178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:47.150616 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:47.150617 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:29:47.150622 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:47.152949 (+ 2327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:47.153967 (+ 1018us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:47.153968 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:47.153968 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:47.154026 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:47.154220 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:47.154220 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:47.154221 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:47.154334 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:47.154382 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:47.154853 (+ 471us) server_negotiation.cc:299] Negotiation successful
0925 07:29:47.154929 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260925 07:29:47.156199 28506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60eb036ef4b842de878eadc22233a7e9: Bootstrap starting.
I20260925 07:29:47.156780 28506 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 60eb036ef4b842de878eadc22233a7e9: Neither blocks nor log segments found. Creating new log.
I20260925 07:29:47.157030 28506 log.cc:824] T 00000000000000000000000000000000 P 60eb036ef4b842de878eadc22233a7e9: Log is configured to *not* fsync() on all Append() calls
I20260925 07:29:47.157967 28602 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80189d3533634041980b5a67abfa7745 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "80189d3533634041980b5a67abfa7745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80189d3533634041980b5a67abfa7745" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 35211 } } peers { permanent_uuid: "60eb036ef4b842de878eadc22233a7e9" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 33707 } } peers { permanent_uuid: "816b53b67748499a96354d14e4696b32" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 40101 } } }
I20260925 07:29:47.158052 28602 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80189d3533634041980b5a67abfa7745 [sys.catalog]: This master's current role is: LEADER
I20260925 07:29:47.158111 28588 sys_catalog.cc:455] T 00000000000000000000000000000000 P 816b53b67748499a96354d14e4696b32 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "80189d3533634041980b5a67abfa7745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80189d3533634041980b5a67abfa7745" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 35211 } } peers { permanent_uuid: "60eb036ef4b842de878eadc22233a7e9" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 33707 } } peers { permanent_uuid: "816b53b67748499a96354d14e4696b32" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 40101 } } }
I20260925 07:29:47.158190 28588 sys_catalog.cc:458] T 00000000000000000000000000000000 P 816b53b67748499a96354d14e4696b32 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 07:29:47.158263 28622 mvcc.cc:205] Tried to move back new op lower bound from 7333156401736478720 to 7333156401724018688. Current Snapshot: MvccSnapshot[applied={T|T < 7333156401736478720}]
I20260925 07:29:47.158572 28626 mvcc.cc:205] Tried to move back new op lower bound from 7333156401736478720 to 7333156401724018688. Current Snapshot: MvccSnapshot[applied={T|T < 7333156401736478720}]
I20260925 07:29:47.159143 28586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:47.142354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41460 (local address 127.5.158.124:40101)
0925 07:29:47.142438 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:47.142439 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:47.143817 (+ 1378us) server_negotiation.cc:407] Connection header received
0925 07:29:47.145027 (+ 1210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:47.145028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:47.145037 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:47.145063 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:47.146173 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:47.146834 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:47.148256 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:47.148560 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:47.148599 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:47.151462 (+ 2863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:47.151463 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:47.151463 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:29:47.151469 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:47.153585 (+ 2116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:47.158235 (+ 4650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:47.158236 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:47.158236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:47.158292 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:47.158551 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:47.158551 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:47.158552 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:47.158645 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:47.158709 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:47.159004 (+ 295us) server_negotiation.cc:299] Negotiation successful
0925 07:29:47.159068 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260925 07:29:47.160682 28434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:47.142762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48250 (local address 127.5.158.126:35211)
0925 07:29:47.142794 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:47.142796 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:47.149633 (+ 6837us) server_negotiation.cc:407] Connection header received
0925 07:29:47.149833 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:47.149834 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:47.149854 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:47.149881 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:47.151030 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:47.151722 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:47.153169 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:47.153455 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:47.153489 (+ 34us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:47.155771 (+ 2282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:47.155772 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:47.155773 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:29:47.155790 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:47.157648 (+ 1858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:47.159487 (+ 1839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:47.159488 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:47.159488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:47.159559 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:47.159922 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:47.159923 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:47.159924 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:47.160077 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:47.160152 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:47.160513 (+ 361us) server_negotiation.cc:299] Negotiation successful
0925 07:29:47.160594 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 07:29:47.161998 28617 catalog_manager.cc:1347] Generated new cluster ID: 37366e7100134807ad391ce265c486c2
I20260925 07:29:47.162045 28617 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 07:29:47.162909 28506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60eb036ef4b842de878eadc22233a7e9: No bootstrap required, opened a new log
I20260925 07:29:47.164827 28506 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60eb036ef4b842de878eadc22233a7e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80189d3533634041980b5a67abfa7745" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 35211 } } peers { permanent_uuid: "60eb036ef4b842de878eadc22233a7e9" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 33707 } } peers { permanent_uuid: "816b53b67748499a96354d14e4696b32" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 40101 } }
I20260925 07:29:47.164937 28506 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60eb036ef4b842de878eadc22233a7e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60eb036ef4b842de878eadc22233a7e9, State: Initialized, Role: FOLLOWER
I20260925 07:29:47.164999 28506 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60eb036ef4b842de878eadc22233a7e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80189d3533634041980b5a67abfa7745" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 35211 } } peers { permanent_uuid: "60eb036ef4b842de878eadc22233a7e9" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 33707 } } peers { permanent_uuid: "816b53b67748499a96354d14e4696b32" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 40101 } }
I20260925 07:29:47.165499 28480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "80189d3533634041980b5a67abfa7745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "60eb036ef4b842de878eadc22233a7e9"
I20260925 07:29:47.165588 28480 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60eb036ef4b842de878eadc22233a7e9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 07:29:47.166306 28480 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 60eb036ef4b842de878eadc22233a7e9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80189d3533634041980b5a67abfa7745 in term 1.
I20260925 07:29:47.166572 28506 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60eb036ef4b842de878eadc22233a7e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 07:29:47.168377 28479 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 60eb036ef4b842de878eadc22233a7e9 [term 1 FOLLOWER]: Refusing update from remote peer 80189d3533634041980b5a67abfa7745: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 07:29:47.168628 28630 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 80189d3533634041980b5a67abfa7745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60eb036ef4b842de878eadc22233a7e9" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 33707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 07:29:47.169467 28509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:47.142603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45522 (local address 127.5.158.125:33707)
0925 07:29:47.142646 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:47.142647 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:47.149186 (+ 6539us) server_negotiation.cc:407] Connection header received
0925 07:29:47.149275 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:47.149276 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:47.149286 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:47.149314 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:47.150588 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:47.151255 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:47.153038 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:47.153330 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:47.153365 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:47.157353 (+ 3988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 07:29:47.157354 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 07:29:47.157355 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 07:29:47.157361 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 07:29:47.158960 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:47.159764 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:47.159765 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:47.159765 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:47.159815 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 07:29:47.160027 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 07:29:47.160029 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 07:29:47.160029 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 07:29:47.160149 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 07:29:47.160237 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:47.169326 (+ 9089us) server_negotiation.cc:299] Negotiation successful
0925 07:29:47.169399 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260925 07:29:47.171960 28643 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 60eb036ef4b842de878eadc22233a7e9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 07:29:47.172062 28643 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 07:29:47.172290 28648 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60eb036ef4b842de878eadc22233a7e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "80189d3533634041980b5a67abfa7745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80189d3533634041980b5a67abfa7745" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 35211 } } peers { permanent_uuid: "60eb036ef4b842de878eadc22233a7e9" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 33707 } } peers { permanent_uuid: "816b53b67748499a96354d14e4696b32" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 40101 } } }
I20260925 07:29:47.173274 28648 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60eb036ef4b842de878eadc22233a7e9 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 07:29:47.172828 28646 mvcc.cc:205] Tried to move back new op lower bound from 7333156401736478720 to 7333156401724018688. Current Snapshot: MvccSnapshot[applied={T|T < 7333156401736478720}]
I20260925 07:29:47.172459 28645 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60eb036ef4b842de878eadc22233a7e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 80189d3533634041980b5a67abfa7745. Latest consensus state: current_term: 1 leader_uuid: "80189d3533634041980b5a67abfa7745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80189d3533634041980b5a67abfa7745" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 35211 } } peers { permanent_uuid: "60eb036ef4b842de878eadc22233a7e9" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 33707 } } peers { permanent_uuid: "816b53b67748499a96354d14e4696b32" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 40101 } } }
I20260925 07:29:47.176108 28645 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60eb036ef4b842de878eadc22233a7e9 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 07:29:47.176411 28648 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60eb036ef4b842de878eadc22233a7e9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "80189d3533634041980b5a67abfa7745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80189d3533634041980b5a67abfa7745" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 35211 } } peers { permanent_uuid: "60eb036ef4b842de878eadc22233a7e9" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 33707 } } peers { permanent_uuid: "816b53b67748499a96354d14e4696b32" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 40101 } } }
I20260925 07:29:47.176731 28630 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80189d3533634041980b5a67abfa7745 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "80189d3533634041980b5a67abfa7745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80189d3533634041980b5a67abfa7745" member_type: VOTER last_known_addr { host: "127.5.158.126" port: 35211 } } peers { permanent_uuid: "60eb036ef4b842de878eadc22233a7e9" member_type: VOTER last_known_addr { host: "127.5.158.125" port: 33707 } } peers { permanent_uuid: "816b53b67748499a96354d14e4696b32" member_type: VOTER last_known_addr { host: "127.5.158.124" port: 40101 } } }
I20260925 07:29:47.176783 28617 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 07:29:47.176805 28630 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80189d3533634041980b5a67abfa7745 [sys.catalog]: This master's current role is: LEADER
I20260925 07:29:47.177351 28617 catalog_manager.cc:1504] Loading token signing keys...
I20260925 07:29:47.177385 28648 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60eb036ef4b842de878eadc22233a7e9 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 07:29:47.192595 28617 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 80189d3533634041980b5a67abfa7745: Generated new TSK 0
I20260925 07:29:47.192847 28617 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 07:29:47.208250 28434 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:29:47.207082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48254 (local address 127.5.158.126:35211)
0925 07:29:47.207116 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:47.207117 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:47.207763 (+ 646us) server_negotiation.cc:407] Connection header received
0925 07:29:47.207849 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:47.207850 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:47.207858 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:47.207880 (+ 22us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:47.208205 (+ 325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48254: BlockingRecv error: recv got EOF from 127.0.0.1:48254 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260925 07:29:47.208820 28509 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:29:47.207358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45538 (local address 127.5.158.125:33707)
0925 07:29:47.207402 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:47.207404 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:47.208427 (+ 1023us) server_negotiation.cc:407] Connection header received
0925 07:29:47.208502 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:47.208502 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:47.208510 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:47.208534 (+ 24us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:47.208776 (+ 242us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45538: BlockingRecv error: recv got EOF from 127.0.0.1:45538 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260925 07:29:47.209393 28586 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:29:47.207600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41470 (local address 127.5.158.124:40101)
0925 07:29:47.207642 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:47.207643 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:47.208989 (+ 1346us) server_negotiation.cc:407] Connection header received
0925 07:29:47.209064 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:47.209065 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:47.209073 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:47.209113 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:47.209346 (+ 233us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41470: BlockingRecv error: recv got EOF from 127.0.0.1:41470 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260925 07:29:47.217968 28509 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:29:47.217318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45544 (local address 127.5.158.125:33707)
0925 07:29:47.217345 (+ 27us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:47.217346 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:47.217509 (+ 163us) server_negotiation.cc:407] Connection header received
0925 07:29:47.217602 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:47.217602 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:47.217615 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:47.217637 (+ 22us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:47.217934 (+ 297us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45544: BlockingRecv error: recv got EOF from 127.0.0.1:45544 (error 108)
Metrics: {"server-negotiator.queue_time_us":13}
W20260925 07:29:47.220646 28509 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:29:47.219969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45552 (local address 127.5.158.125:33707)
0925 07:29:47.219995 (+ 26us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:47.219997 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:47.220204 (+ 207us) server_negotiation.cc:407] Connection header received
0925 07:29:47.220285 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:47.220286 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:47.220294 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:47.220317 (+ 23us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:47.220608 (+ 291us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45552: BlockingRecv error: recv got EOF from 127.0.0.1:45552 (error 108)
Metrics: {"server-negotiator.queue_time_us":13}
W20260925 07:29:47.224323 28509 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:29:47.223609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45558 (local address 127.5.158.125:33707)
0925 07:29:47.223641 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:47.223643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:47.223882 (+ 239us) server_negotiation.cc:407] Connection header received
0925 07:29:47.223973 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:47.223974 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:47.223983 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:47.224010 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:47.224288 (+ 278us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45558: BlockingRecv error: recv got EOF from 127.0.0.1:45558 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260925 07:29:47.230014 28509 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:29:47.229308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45568 (local address 127.5.158.125:33707)
0925 07:29:47.229338 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:47.229340 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:47.229555 (+ 215us) server_negotiation.cc:407] Connection header received
0925 07:29:47.229644 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:47.229645 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:47.229654 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:47.229681 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:47.229977 (+ 296us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45568: BlockingRecv error: recv got EOF from 127.0.0.1:45568 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260925 07:29:47.239636 28509 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:29:47.238968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45580 (local address 127.5.158.125:33707)
0925 07:29:47.238997 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:47.238998 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:47.239199 (+ 201us) server_negotiation.cc:407] Connection header received
0925 07:29:47.239283 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:47.239284 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:47.239292 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:47.239317 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:47.239601 (+ 284us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45580: BlockingRecv error: recv got EOF from 127.0.0.1:45580 (error 108)
Metrics: {"server-negotiator.queue_time_us":14}
W20260925 07:29:47.257319 28509 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:29:47.256555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45588 (local address 127.5.158.125:33707)
0925 07:29:47.256586 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:47.256589 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:47.256835 (+ 246us) server_negotiation.cc:407] Connection header received
0925 07:29:47.256919 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:47.256920 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:47.256928 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:47.256952 (+ 24us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:47.257282 (+ 330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45588: BlockingRecv error: recv got EOF from 127.0.0.1:45588 (error 108)
Metrics: {"server-negotiator.queue_time_us":14}
W20260925 07:29:47.292358 28509 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:29:47.291711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45592 (local address 127.5.158.125:33707)
0925 07:29:47.291765 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:47.291767 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:47.291834 (+ 67us) server_negotiation.cc:407] Connection header received
0925 07:29:47.291924 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:47.291925 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:47.291935 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:47.291992 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:47.292302 (+ 310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45592: BlockingRecv error: recv got EOF from 127.0.0.1:45592 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
W20260925 07:29:47.358096 28509 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:29:47.357471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45604 (local address 127.5.158.125:33707)
0925 07:29:47.357550 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:47.357551 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:47.357578 (+ 27us) server_negotiation.cc:407] Connection header received
0925 07:29:47.357643 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:47.357644 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:47.357653 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:47.357698 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:47.358034 (+ 336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45604: BlockingRecv error: recv got EOF from 127.0.0.1:45604 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260925 07:29:47.487722 28509 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:29:47.487103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32818 (local address 127.5.158.125:33707)
0925 07:29:47.487169 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:47.487171 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:47.487213 (+ 42us) server_negotiation.cc:407] Connection header received
0925 07:29:47.487276 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:47.487277 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:47.487287 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:47.487334 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:47.487640 (+ 306us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32818: BlockingRecv error: recv got EOF from 127.0.0.1:32818 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260925 07:29:47.745533 28509 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 07:29:47.744879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32834 (local address 127.5.158.125:33707)
0925 07:29:47.744936 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:47.744938 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:47.744968 (+ 30us) server_negotiation.cc:407] Connection header received
0925 07:29:47.745035 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:47.745035 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:47.745045 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:47.745156 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0925 07:29:47.745480 (+ 324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32834: BlockingRecv error: recv got EOF from 127.0.0.1:32834 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
I20260925 07:29:48.094547 28599 catalog_manager.cc:1259] Loaded cluster ID: 37366e7100134807ad391ce265c486c2
I20260925 07:29:48.094602 28599 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 816b53b67748499a96354d14e4696b32: loading cluster ID for follower catalog manager: success
I20260925 07:29:48.095453 28599 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 816b53b67748499a96354d14e4696b32: acquiring CA information for follower catalog manager: success
I20260925 07:29:48.095741 28599 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 816b53b67748499a96354d14e4696b32: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 07:29:48.172480 28643 catalog_manager.cc:1259] Loaded cluster ID: 37366e7100134807ad391ce265c486c2
I20260925 07:29:48.172535 28643 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 60eb036ef4b842de878eadc22233a7e9: loading cluster ID for follower catalog manager: success
I20260925 07:29:48.173336 28643 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 60eb036ef4b842de878eadc22233a7e9: acquiring CA information for follower catalog manager: success
I20260925 07:29:48.173612 28643 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 60eb036ef4b842de878eadc22233a7e9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 07:29:48.261209 28715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:48.258543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32850 (local address 127.5.158.125:33707)
0925 07:29:48.258655 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:48.258657 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:48.258740 (+ 83us) server_negotiation.cc:407] Connection header received
0925 07:29:48.258830 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:48.258831 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:48.258887 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:48.258977 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 07:29:48.259126 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:48.259584 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:48.260335 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:48.260556 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:48.260636 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:48.260830 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 07:29:48.260945 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 07:29:48.261033 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:48.261080 (+ 47us) server_negotiation.cc:299] Negotiation successful
0925 07:29:48.261119 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":48,"threads_started":1}
I20260925 07:29:48.265244 28721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:48.262961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51644 (local address 127.5.158.124:40101)
0925 07:29:48.263085 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:48.263086 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:48.263094 (+ 8us) server_negotiation.cc:407] Connection header received
0925 07:29:48.263151 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:48.263151 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:48.263196 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:48.263278 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 07:29:48.263389 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:48.263830 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:48.264481 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:48.264677 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:48.264764 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:48.264904 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 07:29:48.265000 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 07:29:48.265083 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:48.265148 (+ 65us) server_negotiation.cc:299] Negotiation successful
0925 07:29:48.265167 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":51,"threads_started":1}
I20260925 07:29:48.269275 28721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:48.266920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51650 (local address 127.5.158.124:40101)
0925 07:29:48.266952 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:48.266954 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:48.266960 (+ 6us) server_negotiation.cc:407] Connection header received
0925 07:29:48.267047 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:48.267047 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:48.267057 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:48.267083 (+ 26us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 07:29:48.267254 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:48.267870 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:48.268640 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:48.268819 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:48.268888 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:48.269005 (+ 117us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 07:29:48.269028 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 07:29:48.269068 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:48.269171 (+ 103us) server_negotiation.cc:299] Negotiation successful
0925 07:29:48.269189 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 07:29:48.269613 28715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:48.266765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32866 (local address 127.5.158.125:33707)
0925 07:29:48.266797 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:48.266799 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:48.266991 (+ 192us) server_negotiation.cc:407] Connection header received
0925 07:29:48.267047 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:48.267048 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:48.267056 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:48.267098 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 07:29:48.267184 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:48.267839 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:48.268832 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:48.269112 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:48.269146 (+ 34us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:48.269339 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 07:29:48.269366 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 07:29:48.269390 (+ 24us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:48.269509 (+ 119us) server_negotiation.cc:299] Negotiation successful
0925 07:29:48.269527 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 07:29:48.269936 28727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 07:29:48.266789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47624 (local address 127.5.158.126:35211)
0925 07:29:48.266957 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0925 07:29:48.266959 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 07:29:48.267099 (+ 140us) server_negotiation.cc:407] Connection header received
0925 07:29:48.267142 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 07:29:48.267143 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 07:29:48.267190 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 07:29:48.267257 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 07:29:48.267415 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:48.268009 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:48.268910 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 07:29:48.269210 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 07:29:48.269257 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 07:29:48.269452 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 07:29:48.269547 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 07:29:48.269610 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 07:29:48.269730 (+ 120us) server_negotiation.cc:299] Negotiation successful
0925 07:29:48.269750 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":44,"threads_started":1}
I20260925 07:29:48.271615 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 28363
I20260925 07:29:48.276636 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 28438
I20260925 07:29:48.281142 5753 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskROew7s/build/release/bin/kudu with pid 28513
2026-09-25T07:29:48Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2677 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5342 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (277965 ms total)
[ PASSED ] 54 tests.
[ FAILED ] 2 tests, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
[ FAILED ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
2 FAILED TESTS